33med

title: "From PDF to Knowledge Graph"
description: "Discover how 33MED synthesizes unstructured medical literature into an actionable, queryable database using a lean Node.js, MongoDB, and React swarm engine."
date: 2023-10-24
author: "33MED Engineering"
category: "Panoptix Swarm & AI Orchestration"

From PDF to Knowledge Graph: Synthesizing Unstructured Medical Publications into Actionable Databases

The vanguard of computational immunology and surgical robotics isn't just limited by hardwareβ€”it is bottlenecked by data. Every day, thousands of critical biomedical insights are locked away in unstructured PDFs, academic journals, and preprint servers. For 33MED to build genuinely autonomous biotech infrastructure, we need machines that can read, synthesize, and act on this data at scale.

Enter the next evolution of our ImmuNav prototype, powered by the Panoptix Swarm.

Our engineering mandate for Phase 1 was clear: strip out academic infrastructure bloat. No convoluted Python frameworks, no heavy RabbitMQ message brokers, and no multi-layered, fragile database ORMs. Instead, we have architected a highly performant, deterministic, and lean Node.js, MongoDB, and React ecosystem capable of transforming raw biomedical literature into a computable Knowledge Graph.

Here is the technical blueprint of how 33MED achieves real-time data ingestion, deterministic AI orchestration, and autonomous biological simulation.


πŸ—οΈ The Native Node/Mongo Swarm Engine

In distributed systems, complexity is the enemy of reliability. We eliminated dedicated message brokers in favor of utilizing MongoDB as a durable, atomic task queue. By designing a lightweight schema, we track agent states and pipeline jobs natively.

The Atomic Queue Schema

Our architecture relies on two foundational collections: agents and tasks. This design guarantees that every simulation parameterβ€”whether it’s a global directive to halt syndesmophyte growth or a specific molecular targetβ€”is fully auditable.

// Collection: agents
{
  _id: ObjectId("66654a..."),
  role: "BioinformaticsOperator",
  status: "active",
  globalObjective: "Identify molecular targets to halt syndesmophyte growth and spine fusion.",
  humanGuidanceOverride: "Focus explicitly on small-molecule inhibitors targeting IL-17 pathways.",
  lastPulse: ISODate("2026-06-08T22:00:00Z")
}

// Collection: tasks (The Lightweight Queue)
{
  _id: ObjectId("66654b..."),
  agentId: ObjectId("66654a..."),
  tool: "gromacs",
  status: "queued",
  payload: {
    inputFiles: ["target_sequence.fasta"],
    md_params: { nsteps: 50000, dt: 0.002, integrator: "md" } // Validated by Zod
  },
  humanSignOff: { isApproved: true, actionedAt: ISODate(...) },
  logs: []
}

The Background Processing Loop

Within our Node.js Simulation Worker (Port 4000), a recursive polling loop queries MongoDB for actionable, human-approved jobs. It natively spawns child processes for heavy computational CLI tools, appending stdout/stderr logs directly back to the database document in real-time.

async function processQueue() {
  // 1. Global Interrupter Check
  const globalStatus = await db.collection('settings').findOne({ key: 'swarm_status' });
  if (globalStatus?.isPaused) {
    setTimeout(processQueue, 5000);
    return;
  }

  // 2. Atomic fetch of the next approved task
  const task = await db.collection('tasks').findOneAndUpdate(
    { status: 'queued', 'humanSignOff.isApproved': true },
    { $set: { status: 'processing', startedAt: new Date() } },
    { sort: { createdAt: 1 }, returnDocument: 'after' }
  );

  if (!task) return setTimeout(processQueue, 2000);

  try {
    // 3. Native binary execution via Child Process
    const binaryProcess = child_process.spawn(getToolBinary(task.tool), task.payload.cliArgs);
    
    binaryProcess.stderr.on('data', (data) => {
      db.collection('tasks').updateOne({ _id: task._id }, { $push: { logs: data.toString() } });
    });

    binaryProcess.on('close', async (code) => {
      const finalStatus = code === 0 ? 'completed' : 'failed';
      await db.collection('tasks').updateOne({ _id: task._id }, { $set: { status: finalStatus } });
      processQueue(); 
    });
  } catch (error) {
    await db.collection('tasks').updateOne({ _id: task._id }, { $set: { status: 'failed', error: error.message } });
    processQueue();
  }
}

πŸ“‘ The Ingestion Engine: From PubMed to Gemini

Unstructured text is useless to a robotics or computational immunology pipeline. Our Data Ingestion Agents operate as automated background tasks, harvesting raw scientific papers and passing them through Google’s Gemini SDK to extract deterministic JSON payloads.

  1. PubMed Sourcing: Utilizing the NCBI Entrez E-utilities with a registered API key, we fetch literature at 10 requests/second. Target queries are hyper-specific (e.g., Ankylosing Spondylitis AND syndesmophyte).
  2. arXiv Sourcing: We query the native arXiv Atom/XML feed (cat:q-bio.*) for immediate access to cutting-edge preprints without rate-limit constraints.
  3. LLM Extraction Pipeline: Raw XML is injected into Gemini with strict structural instructions:
const systemPrompt = `You are an expert computational immunologist. Analyze the provided scientific text. 
Extract explicit molecular targets, binding affinities, genetic expressions, or pathways related to Ankylosing Spondylitis.
Return the output strictly in the following JSON format:
{
  "targetProtein": "string",
  "associatedCytokines": ["string"],
  "observedEffect": "string",
  "confidenceScore": 0.00
}`;

const result = await geminiModel.generateContent([systemPrompt, rawPaperText]);
const structuredData = JSON.parse(result.response.text());
await db.collection('knowledge_base').insertOne(structuredData);

The result? A deeply interconnected, queryable Knowledge Graph of biological pathways ready for simulation.


🧬 Orchestrating the Core Tool Matrix

Building a commercial SaaS platform requires strict adherence to licensing. We bypass restrictive tools (like AlphaFold 3's non-commercial wrapper) and directly orchestrate open-source alternatives using Node's child_process.


πŸŽ›οΈ Human-In-The-Loop (HITL) Control Center

Autonomous systems must be governed by absolute executive control. The ImmuNav React Frontend (Port 5173) serves as a Swarm Management Dashboard, allowing bioengineers to oversee and intercede without touching configuration files.

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ IMMUNAV AUTOMATION HUB                                [ SYSTEM STATUS: RUNNING ] β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ [ πŸ›‘ GLOBAL SWARM PAUSE ]   [ πŸ”„ RESUME SYSTEM ]   [ ✍️ EDIT GLOBAL OBJECTIVE ] β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ ACTIVE AGENTS                                                                β”‚
β”‚ πŸ€– Agent_Ingestion_1  β”‚ Status: [ WORKING ]  β”‚ Task: Scraping arXiv q-bio    β”‚
β”‚ πŸ€– Agent_Operator_3   β”‚ Status: [ BLOCKED ]  β”‚ Task: Awaiting Config Approvalβ”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ 🚨 AGENT CONFIGURATION APPROVAL GATEWAY                                      β”‚
β”‚ Task ID: #98322  β”‚ Tool Target: GROMACS Molecular Dynamics                    β”‚
β”‚ Target Interaction: Small-molecule inhibitor binding to IL-17 Receptor        β”‚
β”‚                                                                              β”‚
β”‚ [ View Agent-Suggested md_params Payload ]                                   β”‚
β”‚  {                                                                           β”‚
β”‚    "integrator": "md",                                                       β”‚
β”‚    "nsteps": 50000,                                                          β”‚
β”‚    "dt": 0.002                                                               β”‚
β”‚  }                                                                           β”‚
β”‚                                                                              β”‚
β”‚  ✏️ [ Manually Edit JSON Parameters Inline ]                                  β”‚
β”‚  ──────────────────────────────────────────────────────────────────────────  β”‚
β”‚            [ ❌ REJECT CONFIGURATION ]      [ ⚠️ APPROVE & RUN JOB ]          β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ§ͺ Micro-Dataset Testing Protocol: Iterating Safely

Computational physics is resource-heavy. To iterate fast on the swarm engine without exhausting APIs or melting local GPUs, 33MED enforces a strict Micro-Dataset Testing Protocol:

  1. Sourcing Sandboxes: We hardcode boundaries during development (&retmax=3 for PubMed; &max_results=3 for arXiv). This ensures our regex sanitizers and Gemini extraction work flawlessly before opening the data floodgates.
  2. Molecular Sandboxes: We do not test Chai-1 or GROMACS on complex human spinal proteins initially. We use Hen Egg White Lysozyme (PDB ID: 1AKI), a standard benchmarking structure. Energy minimization on Lysozyme takes less than two minutes, providing rapid validation that our Node runtime processes exits codes and stdout strings cleanly.
  3. Mechanical Sandboxes: Instead of loading full biomechanical skeletal rigs, we utilize simple single-joint pendulums or 2D cellular grid templates found in the OpenSim and Chaste repositories to test our React dashboard's rendering of thermodynamic charts and XML generation.

By transforming PDFs into a queryable Knowledge Graph, and linking that graph directly to deterministic, open-source computational execution, 33MED is not just cataloging biomedical dataβ€”we are building the operating system for autonomous biotechnology.

Welcome to Phase 1.