From RAG to Metagraph: Why Semantic Similarity Hit a Wall

Watch enough GraphRAG demos and read enough "agentic memory" threads, and the autocomplete-with-a-vector-store trick, performed sixty different ways by sixty different teams, turns out to be the same trick every time. Chunk a document. Embed the chunks. Store the vectors. Retrieve by cosine similarity. Shove the top results into a prompt. Pray.

Pray that the right chunk shows up. Pray that the model doesn't hallucinate the parts that didn't. Pray that when three researchers disagree on something fundamental, your system doesn't average their views into a confident-sounding mush. Pray that next week's information doesn't quietly contradict last week's, with neither flagged for anyone's attention. That's the state of enterprise RAG in 2026. It's a useful technology, and for narrow corpora with clean documents and simple queries it's approximately sufficient. But you can't bet a multi-domain agentic platform on "approximately sufficient", because the failure modes are structural, not random.

Vector retrieval has four structural failures, and escaping them takes one architectural shift, from retrieval to reasoning over structure. The shift moves you from a flat cloud of embeddings to a metagraph, a graph where edges themselves can be targets of other edges, where every fact carries its provenance and its truth value, and where time is a first-class property rather than a metadata afterthought.

What semantic similarity actually does

Start with the mechanism, because most of the failure modes follow directly from it. When you embed a passage of text into a vector, you've projected it, in strict mathematical terms, into a high-dimensional space where "semantic similarity" is operationally defined as cosine proximity. Two passages that talk about similar things, in similar registers, with overlapping vocabulary and structure, will end up near each other. That's the entire trick, and it's a beautiful one, but it's also a profoundly impoverished representation of what knowledge is.

Consider what is not in the vector. The vector doesn't know who said the thing it represents. It doesn't know when. It doesn't know whether the speaker has since revised their position. It doesn't know whether three other sources say the opposite. It doesn't know whether the claim is empirical fact, scientific consensus, expert opinion, or speculative gesture. It doesn't know what other claims this one logically depends on, or contradicts, or refines. All of that, the information humans use to reason about claims, is either stripped during embedding or shoved into a metadata column the retrieval pipeline doesn't consult.

The result is a retrieval system that is brilliant at finding text that looks like the answer and entirely blind to whether the text it found is the answer. Most of the time, in narrow domains, the distinction doesn't bite, because looking like the answer and being the answer are correlated enough that the system produces useful output. But the correlation breaks down at the moments when correct reasoning matters most: contested topics, temporally evolving positions, multi-hop questions, and claims whose validity depends on the speaker.

Failure mode one: the averaging problem

When multiple documents say different things about the same topic, cosine similarity retrieves all of them. It can't do otherwise, because they all look semantically close to the query. The top-K results are then concatenated into the model's context window, and the model is asked to generate a coherent answer. It has contradictory evidence and no structural guidance about how to weigh it, so it generates the average. It produces text that splits the difference between the positions, hedges where it should pick a side, and confidently asserts a synthesis that no actual source would endorse.

The failure lives in the representation. The model has no way to know that source A is a peer-reviewed meta-analysis and source B is a single blog post, because both were tokenized into context with equal weight. It has no way to know that the field has converged on A and abandoned B since 2022, because the embeddings are atemporal. The averaging is, structurally, the only thing the model can do. The system was built to make it do that.

The metagraph approach inverts the equal-weight assumption. When two claims contradict, the contradiction is itself a structural fact in the graph. There is a Controversy node that explicitly holds the competing positions, links them to their sources with credibility weights, marks them with temporal validity windows, and stores a computed resolution_status. When an agent queries on the topic, it receives the controversy, the positions, the weights, and a calibrated representation of how confident the system is in each, not an averaged answer. The agent can then decide whether to present the disagreement, take a position with appropriate hedging, or escalate. The reasoning is preserved because the structure preserved it.

Failure mode two: context blindness

Semantic similarity doesn't know what year it is. The vector that represents "Karpathy believes scaling is all you need", indexed in 2019, will still surface in 2026 as a relevant match for "what does Karpathy think about scaling?", even though Karpathy himself has spent the intervening seven years walking that claim back, qualifying it, refining it, and finally largely abandoning it in favor of structured world models. The embedding doesn't age. The cosine distance doesn't decay. The system happily retrieves the seven-year-old position because the vocabulary still matches.

That's the context-blindness problem. Standard RAG has no native mechanism for "this was true then, this is true now." You can bolt on metadata filters, sure, or try to rerank by date. Teams do both, and it helps, and it's also a thin tarpaulin draped over a structural hole. The hole is that the retrieval system pushes time back onto you. It asks you to manage temporal validity yourself, badly, with whatever bookkeeping you can hold in your head.

A bi-temporal knowledge graph, by contrast, treats time as a primitive. Every claim has two timestamps: t_event (when the event the claim is about actually happened) and t_ingestion (when the claim entered your system). Every claim has a validity window, a valid_from and a valid_to. When a new claim supersedes an old one, the old claim isn't deleted. Its valid_to is closed. It remains in the graph, queryable as historical state, but no longer returned by present-tense queries. Karpathy 2019 and Karpathy 2024 both exist. Both are honored. Neither is averaged into a fictional Karpathy 2021 who never spoke.

Closing a validity window sounds like a small thing, and it's the difference between an agent that says "Karpathy believes scaling is all you need" with full confidence (because it found a 2019 transcript) and an agent that says "Karpathy's 2019 position emphasized scaling; he has since revised this view, and as of 2024 emphasizes structured world models." The first agent is wrong and the second is right. The difference is bi-temporal modeling, and it's invisible to any system whose substrate is a flat vector cloud.

Failure mode three: attribution loss

By the time a passage of text has been chunked, embedded, retrieved, and concatenated into the model's context, it has been stripped of everything except its words. The provenance is gone. The source credibility is gone. The publication date is gone. The model sees "the optimal learning rate for this architecture is approximately 3e-4" and has no idea whether it came from the foundational paper, a Stack Overflow answer, or a tweet from someone who's angry on the internet. They're all just context tokens now.

That's the attribution-loss problem, and it's the failure mode that quietly destroys most production RAG systems over time. In the early days, when the corpus is small and curated, the sources are all decent and the loss of attribution doesn't bite. But every corpus grows. Sources get added. Quality drifts. Eventually some piece of low-quality content gets in, matches well on cosine similarity, and the system confidently surfaces it as if it were the foundational paper. There's no defense, because the system can't tell the difference.

In a metagraph, provenance is structural. A claim lives as a node, linked by SOURCED_FROM edges to the source that asserted it, linked by ASSERTED_AT edges to a timestamp, linked by CREDIBILITY meta-edges to the source's track record. When the agent retrieves the claim, it retrieves the whole structural neighborhood. The credibility weights are visible. The source's history is queryable. The agent can ask, structurally, "how often has this source been right about claims in this domain?" and weight accordingly. The provenance is the same kind of object the claim is, sitting in the same graph and carried by the same structure.

And when the source's credibility shifts, when a researcher's predictions prove repeatedly correct, or repeatedly wrong, the weight propagates. The next time their claim surfaces in a query, the system already knows. That propagation is reinforcement learning at the level of the world model itself, not at the level of weights inside a transformer. It's structural learning, and it accumulates.

Failure mode four: hop blindness

The fourth failure mode is the one that bites hardest in interesting questions. "Which researchers worked with Hinton at U of T who later went to OpenAI?" No single document contains this answer. Answering it requires composing three facts: who worked with Hinton at U of T, who went to OpenAI, and the intersection. Vector retrieval can find documents about Hinton, documents about OpenAI alumni, and documents about U of T, but the composition isn't in the embedding space, because composition is a structural operation, not a similarity operation.

You can sometimes paper over this with multi-step prompting. Ask the model to break the question into sub-queries, retrieve for each, then synthesize. The approach works occasionally. It's brittle and expensive, and its failure modes are silent. The model decides what sub-queries to issue based on what looks plausible, the retrieval steps each lose information, and the synthesis happens at the end with all the same averaging problems we just discussed. The system can sometimes answer the question, but it can't be relied on to answer it.

A knowledge graph answers this kind of question natively, because the graph is the composition. A single Cypher query traverses from the Hinton node along WORKED_WITH edges to other researchers, then along EMPLOYED_BY edges to OpenAI, with a timestamp filter on each. The answer arrives as a structural traversal that's fast, correct, and auditable. It also composes cleanly with the other three properties we've been building: temporal filtering, contradiction handling, and source weighting all apply at every step.

RAG with embeddings is a bicycle, a perfectly good thing to have for trips around the block. Structural reasoning over a temporal, epistemic, provenance-rich metagraph is the car, and it's what you need when the road actually goes somewhere.

What you give up to get here

Going from RAG to metagraph isn't free, and the people who tell you it is are selling something. The costs are real, and pretending otherwise is the kind of mistake that gets sober engineers fired by less-sober ones.

You give up simplicity of ingestion. Chunking a document and embedding the chunks is a one-line script. Extracting entities, resolving them against an ontology, identifying claims, classifying claim types, detecting contradictions with existing graph state, and attaching provenance add up to a serious pipeline. It needs NLP infrastructure, it needs ontology management, it needs human review for ambiguous cases, and it needs continuous tending as the corpus grows. The ingestion side of a metagraph is, conservatively, an order of magnitude more engineering than the ingestion side of a vector store.

You give up generality of retrieval. Vector retrieval works on any kind of text. You can throw a PDF, a tweet, and a recipe at it and get something back. Graph retrieval requires that the content has been parsed into the graph's schema first. The schema is your friend, because it's what enables the structural reasoning, but it's also a commitment. Adding a new content type means adding new parsers, new entity types, new relations. The system is more powerful and less universal. That's a real trade.

You give up, in some sense, the illusion of zero-cost knowledge. Vector stores let you pretend that having a pile of documents is the same as having a knowledge base. The pile is only the raw material. The knowledge base is what you get after you've done the work of structuring that raw material, and a vector store hasn't done that work. It has only made the pile faster to search. The metagraph approach names the work and pays for it: the work is real, the value is real, and there's no shortcut.

Why now

Three things have changed in the last eighteen months that make the metagraph approach newly viable as a production strategy rather than a research curiosity. First, large language models have become reliable enough at entity extraction and relation extraction that the ingestion pipeline can be built mostly on top of LLM calls rather than fragile classical NLP. That shift removes a category of brittleness that used to make knowledge graphs prohibitively expensive to maintain.

Second, the infrastructure has matured. Neo4j with APOC, SurrealDB's multi-model engine, and Graphiti's bi-temporal abstractions on top of Neo4j are production-grade tools that didn't exist, or didn't work well enough, three years ago. Today a small team can stand up a temporal knowledge graph with metagraph-style epistemic patterns and have it serve real queries at real latencies. That wasn't true in 2022.

Third, and most importantly, the agentic frontier has revealed what flat retrieval can't do. Every team building serious agents (agents that take actions, accumulate memory, reason across multiple turns, and handle contested topics) has hit the wall we described above. Some have papered over it with elaborate prompting. Some have given up and shipped narrower products. The teams that are going to win the agentic decade are the ones who recognized the wall is structural and started building the substrate that gets through it. The window to start that build is open right now, and it won't be open forever.

The shape of the substrate

The substrate is a metagraph: a graph where edges can target edges, where every fact has a truth value, where time is a primitive, and where reasoning is structural rather than statistical. Concretely, in production today, this looks like Graphiti on top of Neo4j for the temporal graph core, with SurrealDB sitting alongside as a unified multi-model layer for vector and full-text retrieval, with a custom contradiction engine running continuously over the graph, with a confidence propagation system (inspired by Probabilistic Logic Networks, the reasoning system Ben Goertzel built for OpenCog) updating truth values as new evidence arrives, and with the whole thing exposed to agents via three primitive operations: query, assert, and update_belief.

Each of those design decisions gets a dedicated essay in this series. One principle holds the design together: knowledge is structure, not similarity. Two facts can be syntactically identical and epistemically different. They can be cosine-near and structurally opposed. They can look the same to an embedding and behave the same to an embedding and be different facts about different things, said by different people, at different times, with different weights. Only a substrate that holds that difference structurally can reason with it. And only an agent that reasons structurally can be trusted with anything that matters.

What this series will cover

Each of the twelve essays drills into one architectural decision that distinguishes a metagraph-based knowledge system from the RAG-shaped systems most teams are building today, starting with bi-temporal modeling in essay two and ending with second-level thinking for infrastructure bets in essay twelve.

One thesis runs under all twelve. Teams that treat knowledge as structure will produce agents that reason, and teams that treat knowledge as similarity will produce agents that retrieve. Both kinds of agents have markets, but only one has a future as the substrate for everything else. The bet we're making with Wiki Design Co. is that the structural kind is the substrate, and that the right time to build it is right now, before everyone else figures out the same thing and the structural moat closes behind us, with the years of accumulated episodic data, the careful schema decisions, and the contradiction history all compounding.

The next essay starts where this one leaves off, with time: what it means for a fact to be true at one moment and not at another, and the bi-temporal model that lets a knowledge system hold that distinction without averaging it into noise.

---

END OF ESSAY 01 · CONTINUE TO ESSAY 02 →