Production-Grade · Neo4j · Multi-Agent LLM

Knowledge Graph
Intelligence System

A production-grade knowledge intelligence system that replaces expensive LLM-only reasoning with an ontology-driven Neo4j knowledge graph. Unstructured data is ingested at 10,000+ records per second, converted into validated semantic triplets by a multi-agent LLM pipeline, and stored in a continuously evolving ontology that supports low-cost, high-precision reasoning for downstream agents.

●400M+ triplets ingested ●10,000+ records/sec throughput ●50M+ node Neo4j graph ●3-layer LLM agent pipeline ●~6.5x cheaper agentic queries ●$140K/year saved

Python Neo4j Apache Airflow Docker LangGraph Ollama Multi-LLM Vector Embeddings Cypher

Architecture

Pipeline & Methodology

400M+

Semantic triplets ingested

10K+

Records per second throughput

50M+

Nodes and edges in the graph

~6.5x

Cheaper agentic queries

Step 01: Ingestion

Distributed Data Pipeline

A horizontally scalable ingestion pipeline built with Python, Apache Airflow, and Docker processes unstructured data sources at over 10,000 records per second. Both batch and streaming sources are supported. The pipeline handles fault tolerance and automatic retry with no manual intervention required.

Step 02: Normalization

Entity and Schema Alignment

Every incoming record passes through entity normalization (coref resolution, alias deduplication), schema alignment against the current ontology schema, and ontology-aware validation. This prevents duplicate nodes and maintains semantic consistency as the graph scales to tens of millions of nodes.

Step 03: Extraction

Parent Agent Dispatches Child Extractors

A parent agent receives each normalized record and dispatches it simultaneously to multiple domain-scoped child extraction agents. Each child is specialized for a subset of relation types, keeping context windows small. The parent collects all extracted (subject, predicate, object) triplets and deduplicates overlapping outputs before passing them to scoring.

Step 04: Voting Session

Multi-Agent Vote on Ontology Score

Each extracted triplet enters a voting session: multiple child agents independently evaluate the relation's plausibility, cast a vote (accept or reject), and submit a confidence score between 0 and 1. The parent agent aggregates all votes using a weighted consensus to compute the optimal ontology score, which becomes the edge weight stored in Neo4j.

Step 05: Parent Decision

Conflict Resolution and Commit

The parent agent reviews voting results. Strong consensus (majority agreement above threshold) triggers an immediate commit to the graph. Split votes send the relation to a conflict staging queue where the parent enforces domain rules to break the tie. Every decision, accepted or rejected, includes traceable per-agent justifications for full explainability.

Step 06: Reasoning

Hybrid Graph and Vector Query

Downstream agents query via Cypher graph traversal for multi-hop reasoning, causal inference, and domain logic, and via vector embeddings for semantic similarity and paraphrase matching. The hybrid layer shifts semantic lookup cost from LLM inference to deterministic graph traversal.

Ontology Evolution

Unlike static knowledge bases, this system treats the ontology as a living schema. When new data arrives, the agent pipeline extends existing concepts, adds new relation types, and re-weights existing edges without requiring manual schema updates. The graph grows more accurate over time.

Why Not Vector-Only

Pure vector search cannot encode explicit hierarchies, constraints, or causal chains. The ontology captures that "A is a subclass of B", "X causes Y", and "Z is incompatible with W" as first-class graph edges, enabling reasoning that embedding similarity alone cannot support.

Why Voting Beats a Single Judge

A single LLM judge can hallucinate consistently in one direction. The voting session forces independent models to disagree openly before a consensus is reached. Correlated errors are rare across heterogeneous models, so the aggregate ontology score is more reliable than any individual agent's output.

System Design

Architecture Diagram

End-to-End System Flow

Parent-Child Architecture with Voting Session

Semantic Layer

The Ontology Model

The ontology is the contract the whole system is built on. It is not the data itself, but the schema of meaning: the set of allowed entity types (classes), the relation types permitted between them, and the constraints that keep the graph internally consistent. Every triplet the agents propose is validated against this schema before it is ever written to Neo4j. The example below uses a pharmaceutical domain, the same domain the downstream agents reason over.

Definition

Ontology vs Instance Graph

The ontology (T-Box) describes the types: "a Drug can TREAT a Disease", "a Mechanism INHIBITS a Target". The instance graph (A-Box) holds the actual facts: "Atorvastatin TREATS Hypercholesterolemia". Separating the two lets the schema stay small and stable while the instance graph grows to 50M+ nodes.

Why It Matters

Constraints Prevent Garbage Edges

Because TREATS is typed as Drug → Disease, an agent that tries to write "Disease TREATS Drug" is rejected at validation time, not discovered later. The ontology turns a class of hallucinations into a schema violation the pipeline can catch deterministically.

Ontology schema — classes, typed relations, constraints

# ── Classes (node labels) ──────────────────────────────
Drug   DrugClass   Mechanism   Target   Disease   AdverseEvent

# ── Relation types (edge types) with domain → range ───
(Drug)       -[:SUBCLASS_OF]->     (DrugClass)
(Drug)       -[:HAS_MECHANISM]->   (Mechanism)
(Mechanism)  -[:INHIBITS]->        (Target)
(Drug)       -[:TREATS]->          (Disease)
(Drug)       -[:CAUSES]->          (AdverseEvent)
(Drug)       -[:INTERACTS_WITH]->  (Drug)   // symmetric

# ── Constraints enforced before commit ────────────────
CONSTRAINT treats_typing     : TREATS valid only Drug → Disease
CONSTRAINT mechanism_required : every Drug has ≥ 1 HAS_MECHANISM
CONSTRAINT symmetry           : INTERACTS_WITH(a,b) ⇒ INTERACTS_WITH(b,a)

Instance graph — validated facts written as edges

(Atorvastatin)                  -[:SUBCLASS_OF]->    (Statin)
(Atorvastatin)                  -[:HAS_MECHANISM]->  (HMG-CoA reductase inhibition)
(HMG-CoA reductase inhibition)  -[:INHIBITS]->       (HMG-CoA reductase)
(Atorvastatin)                  -[:TREATS]->         (Hypercholesterolemia)
(Atorvastatin)                  -[:CAUSES]->         (Myopathy)

Ontology Schema (T-Box) and a Validated Instance (A-Box)

Multi-Hop Query Becomes a Cypher Traversal

"Which drugs share a mechanism with Atorvastatin and also treat Hyperlipidemia?" is three hops in the ontology: Drug → Mechanism → Drug → Disease. Because the relation types are typed, this compiles into a single deterministic Cypher path query rather than a chain of LLM guesses.

The Ontology Is What the Voting Session Protects

The parent-child voting session shown above is not voting on free text, it is voting on whether a proposed edge is type-valid and plausible under this schema. The ontology gives the agents an objective rubric, which is what makes the consensus score meaningful.

Grounding Layer

The Lexical Graph

A knowledge graph that only stores structured triplets can tell you what it believes, but not where it read it. The lexical graph is the second, coupled layer that fixes this. It holds the source text itself, broken into ordered chunks, and links every chunk to the entities it mentions. The domain graph answers "what is true"; the lexical graph answers "which passage proves it." Together they are what makes the system a GraphRAG system rather than just a graph.

Step 01: Segment

Document → Section → Chunk

Each source document is split into Document → Section → Chunk nodes of roughly 200–400 tokens. Sequential NEXT edges preserve reading order so neighbouring context can be recovered later without re-parsing the file.

Step 02: Embed

One Vector per Chunk

Every Chunk node carries a stored embedding. This is the only place vectors live: on the text, not on the facts. It keeps the domain graph clean while still allowing fuzzy, out-of-ontology recall when a query has no exact entity match.

Step 03: Link

MENTIONS & DERIVED_FROM

The extraction agents emit two provenance edges: Chunk -[:MENTIONS]-> Entity with character offsets, and Triplet -[:DERIVED_FROM]-> Chunk. Every fact in the domain graph is now traceable back to the exact passage that produced it.

Two Coupled Layers — Lexical (text) Anchored to Domain (facts)

What the Lexical Graph Buys You

Citations for free. Because each answer is assembled from specific chunks, the system returns the exact source passage alongside every claim. It also gives the voting agents something concrete to check a proposed edge against, and lets retrieval fall back to vector search on chunks when a query has no clean entity match.

Two Entities, Connected Through Text

Two chunks that mention the same entity become 2-hop connected through that entity. This is what lets the system retrieve a tight, connected neighbourhood of evidence instead of a flat bag of top-k chunks, and it is the structural reason the token footprint stays small.

Retrieval Strategy

Graph Agentic RAG vs Normal Agentic RAG

Both approaches put an LLM agent in a loop with a retriever. The difference is where the multi-hop reasoning happens. Normal agentic RAG makes the LLM do the stitching across many retrieved chunks, paying for it in tokens and iterations. Graph agentic RAG pushes the multi-hop step into the graph engine (deterministic, no tokens) and uses the LLM only to phrase the final answer over a small, pre-assembled context. The numbers below are illustrative per-query figures for a typical 3-hop question, not marketing headlines.

Where the Computation Goes: Two Retrieval Loops Side by Side

Worked example — the actual Atorvastatin query, real tokens and dollars

WORKED EXAMPLE  ·  query: "Which drugs share a mechanism with
Atorvastatin and also treat Hyperlipidemia?"   — a 3-hop question
Model: GPT-4o    $2.50 / 1M input tok    ·    $10.00 / 1M output tok

── NORMAL AGENTIC RAG ──────────────────────────────────────
  vector top-k = 15 chunks (~400 tok ea) → 6,000 tok context
  ReAct loop, 3 LLM turns (one re-retrieval):
    turn 1    in  6,530   out 200    read 15 chunks + reason
    turn 2    in  9,930   out 200    re-retrieve 8 + reason
    turn 3    in 10,130   out 400    synthesize answer
    ───────────────────────────────────────────────
    input  26,590 tok × $2.50/1M  =  $0.0665
    output    800 tok × $10/1M    =  $0.0080
    TOTAL                         ≈  $0.074  / query

── GRAPH AGENTIC RAG ───────────────────────────────────────
  entity-link "Atorvastatin", "Hyperlipidemia" → anchor nodes
  Cypher (runs in the DB, 0 LLM tokens, ~8 ms):
    MATCH (d:Drug {name:"Atorvastatin"})-[:HAS_MECHANISM]->(m)
          <-[:HAS_MECHANISM]-(peer:Drug)-[:TREATS]->
          (:Disease {name:"Hyperlipidemia"})  RETURN peer
  pull 4 proof chunks via DERIVED_FROM
  1 synthesis LLM call:
    input   2,430 tok  (400 sys + 30 q + 400 subgraph + 1,600 chunks)
    output    450 tok
    ───────────────────────────────────────────────
    input  2,430 tok × $2.50/1M   =  $0.0061
    output   450 tok × $10/1M     =  $0.0045
    TOTAL                         ≈  $0.011  / query

── RESULT ──────────────────────────────────────────────────
  $0.074 → $0.011    =  ~6.5× cheaper,  ~$0.063 saved / query
  26,590 → 2,430 input tokens    =  ~11× fewer tokens
  the multi-hop join ran in the graph engine, not in the LLM

Summary — same query, side by side

Dimension	Normal Agentic RAG	Graph Agentic RAG
Where multi-hop runs	Inside the LLM, across iterations (probabilistic)	In the graph engine via Cypher (deterministic)
Retrieval breadth	15 chunks by similarity, redundant	4 chunks, exactly the connected subgraph
LLM calls / query	3–5 (ReAct loop)	1 synthesis (+ optional verify)
Embedding calls	1 query + re-embeds on re-retrieval	1 query embed for anchoring
Retrieved context	15 × ~400 = 6,000 tok	400 (subgraph) + 4 × ~400 = 2,000 tok
Context re-sent per loop	Yes, grows each iteration	No, single pass
Input tokens / query	≈ 26,600	≈ 2,430
Cost / query · GPT-4o	≈ $0.074	≈ $0.011
Effective reduction	~6.5× cheaper · ~11× fewer input tokens · ~$0.063 saved / query · ~4× fewer LLM calls

Honest Blended Numbers

Not every query is a 3-hop question. Across a mixed production workload (many one-hop lookups plus some deep traversals), the blended effect is roughly $0.03–0.05 saved per query and 3–5× fewer tokens. The ~$0.063 / ~6.5× figure above is the multi-hop case, which is exactly where normal RAG burns the most tokens stitching chunks together.

Pure Lookups Are Effectively Free

A pure factual lookup ("what mechanism does Atorvastatin act on?") resolves entirely in Cypher with no LLM call at all — a fraction of a cent of database compute, effectively free next to even a small LLM answer. Only questions that need natural-language synthesis spend the ~$0.011 above. Routing the easy majority to pure graph queries is where the biggest savings actually come from.

Why Fewer Tokens Also Means Fewer Errors

A 6,000-token bag of similar chunks gives the model many chances to latch onto a near-miss passage. A 2,100-token, graph-selected context contains only nodes that are actually connected to the query, so the model has less room to hallucinate and every claim already carries a provenance edge for verification.

Compute, Not Just Dollars

Fewer input tokens and fewer round-trips means less GPU time per answer and lower memory pressure on the serving layer. At fleet scale this raises queries-per-GPU throughput, which is what actually caps how many agents you can run concurrently, independent of the per-query cost line.

Query Layer

Hybrid Reasoning Architecture

Symbolic Graph Reasoning

Multi-hop traversal across entity chains
Explicit hierarchies (is-a, part-of, causes)
Domain constraint enforcement
Causal inference via directed edges
Deterministic, zero-LLM-cost at query time
Full explainability via graph path

Semantic Vector Search

Paraphrase and synonym matching
Fuzzy concept retrieval
Semantic similarity scoring
Handles out-of-ontology queries
Embedding-based entity linking
Flexible recall for novel concepts

Why Graph Traversal is Cheaper

A full agentic answer over vector-retrieved chunks runs ~$0.074 on GPT-4o (~26,600 input tokens across a 3-turn loop). The same answer over the graph is one synthesis call on a graph-selected context: ~$0.011. At ~6,000 agentic queries/day across the fleet (~2.2M/year), the ~$0.063 per-query gap compounds to ~$139K/year, before counting the pure lookups that skip the LLM entirely.

Multi-Hop Reasoning Example

"Which drugs share a mechanism with Drug X and treat Disease Y?" This requires traversing three hops: Drug X to mechanism, mechanism to related drugs, related drugs to diseases. A pure LLM must hallucinate or retrieve. The graph answers deterministically in milliseconds via Cypher.

Results

Cost Impact & Performance

Query Cost: Normal Agentic RAG vs Graph Agentic RAG · GPT-4o pricing

~$139,000 / year

Derived bottom-up: ~$0.063 saved per agentic query × ~2.2M queries/year (~6,000/day across the fleet). Pure factual lookups that resolve in Cypher with no LLM call push the real figure higher.

400M+ triplets

Total semantic triplets ingested and validated through the pipeline. Each triplet represents a structured (subject, predicate, object) fact stored as a graph edge.

Full explainability

Every accepted or rejected relation includes a traceable justification from the judge agent. Reasoning paths are inspectable as graph traversal paths, not black-box outputs.

Zero semantic drift

Multi-LLM governance prevents hallucinated edges from entering the graph. Relations that conflict with existing ontology rules are staged and resolved before storage.

Horizontal scaling

The Airflow + Docker ingestion layer scales by adding worker containers. Throughput scales linearly with added workers, with no single point of failure.

Self-correcting ontology

As more data flows through the pipeline, the ontology expands and existing edge weights are recalibrated. The graph becomes more accurate over time without requiring manual schema maintenance.

Implementation

Technology Decisions

Storage: Neo4j

Why a Property Graph Over a Relational DB

Relational databases require expensive JOIN chains for multi-hop queries. Neo4j stores relationships as first-class objects, so traversing 3 hops across 50M nodes takes the same time as traversing 3 rows. Pointer traversal replaces index scans. The Cypher query language also maps naturally to ontology concepts.

Orchestration: Apache Airflow

Why Airflow Over a Simple Script

The ingestion pipeline handles hundreds of millions of records across multiple source formats and schedules. Airflow provides DAG-based dependency management, automatic retry, task-level failure isolation, and a full audit trail of every ingestion run. Scaling a script to production requires inventing all of this.

Agent Framework: LangGraph + Ollama

Parent-Child with Voting via LangGraph

LangGraph manages the parent-child state machine: the parent node fans out tasks to child nodes in parallel, waits for all votes to return, then runs the aggregation node to compute the optimal ontology score. Ollama hosts heterogeneous open-source models locally so each child agent runs a different model, preventing correlated hallucination across the voting session.

Hybrid Query: Graph + Vectors

Why Not Graph Alone

The ontology is strong on known facts but brittle on novel or ambiguous queries. Vector embeddings provide the semantic flexibility to match paraphrases and retrieve concepts not yet in the ontology. The hybrid layer uses the graph for precision and vectors for recall, combining their strengths into a single query interface.

Knowledge Graph Intelligence System