From RAG to Metagraph: Why Semantic Similarity Hit a Wall

Here is the situation, stated plainly so we don't have to dance around it for the next four thousand words. You have watched every GraphRAG demo. You have read every "agentic memory" thread. You have seen the autocomplete-with-a-vector-store trick performed sixty different ways by sixty different teams, and you have started to notice that the trick is always the same trick. Chunk a document. Embed the chunks. Store the vectors. Retrieve by cosine similarity. Shove the top results into a prompt. Pray.

Pray that the right chunk shows up. Pray that the model doesn't hallucinate the parts that didn't. Pray that when three researchers disagree on something fundamental, your system doesn't average their views into a confident-sounding mush. Pray that next week's information doesn't quietly contradict last week's, with neither flagged for anyone's attention. This is the state of enterprise RAG in 2026. It is not nothing, it is a genuinely useful technology, and for narrow corpora with clean documents and simple queries it is approximately sufficient. The trouble is that "approximately sufficient" is not a thesis you can bet a multi-domain agentic platform on, because the failure modes are structural, not random.

Vector retrieval has four structural failures. What follows takes each one to its cause and to the architectural shift required to escape it. The shift, in one phrase, is from retrieval to reasoning over structure. From a flat cloud of embeddings to a metagraph, a graph where edges themselves can be targets of other edges, where every fact carries its provenance and its truth value, where time is a first-class property rather than a metadata afterthought. The metagraph is a different kind of substrate, and once you see it you cannot unsee it.

What semantic similarity actually does

Start with the mechanism, because most of the failure modes follow directly from it. When you embed a passage of text into a vector, what you have done, in strict mathematical terms, is project that text into a high-dimensional space in which "semantic similarity" is operationally defined as cosine proximity. Two passages that talk about similar things, in similar registers, with overlapping vocabulary and structure, will end up near each other. That is the entire trick. It is a beautiful trick. It is also a profoundly impoverished representation of what knowledge actually is.

Consider what is not in the vector. The vector does not know who said the thing it represents. It does not know when. It does not know whether the speaker has since revised their position. It does not know whether three other sources say the opposite. It does not know whether the claim is empirical fact, scientific consensus, expert opinion, or speculative gesture. It does not know what other claims this one logically depends on, or contradicts, or refines. All of that information, the information that humans actually use to reason about claims, is either stripped during embedding or shoved into a metadata column the retrieval pipeline does not consult.

The result is a retrieval system that is brilliant at finding text that looks like the answer and entirely blind to whether the text it found is the answer. Most of the time, in narrow domains, this distinction does not bite. The looking-like and the being-the are correlated enough that the system produces useful output. But the correlation breaks down at exactly the moments when correct reasoning matters most: contested topics, temporally evolving positions, multi-hop questions, and claims whose validity depends on the speaker.

Failure mode one: the averaging problem

When multiple documents say different things about the same topic, cosine similarity retrieves all of them. It cannot do otherwise, they all look semantically close to the query. The top-K results are then concatenated into the model's context window, and the model is asked to generate a coherent answer. What the model does, faced with contradictory evidence and no structural guidance about how to weigh it, is generate the average. It produces text that splits the difference between the positions, hedges where it should pick a side, and confidently asserts a synthesis that no actual source would endorse.

The failure lives in the representation. The model has no way to know that source A is a peer-reviewed meta-analysis and source B is a single blog post, because both were tokenized into context with equal weight. It has no way to know that the field has converged on A and abandoned B since 2022, because the embeddings are atemporal. The averaging is, structurally, the only thing the model can do. The system was built to make it do that.

The metagraph approach inverts the assumption. When two claims contradict, the contradiction is itself a structural fact in the graph. There is a Controversy node that explicitly holds the competing positions, links them to their sources with credibility weights, marks them with temporal validity windows, and stores a computed resolution_status. When an agent queries on the topic, it does not receive an averaged answer, it receives the controversy, the positions, the weights, and a calibrated representation of how confident the system is in each. The agent can then decide whether to present the disagreement, take a position with appropriate hedging, or escalate. The reasoning is preserved because the structure preserved it.

Failure mode two: context blindness

Semantic similarity does not know what year it is. The vector that represents "Karpathy believes scaling is all you need", indexed in 2019, will still surface in 2026 as a relevant match for "what does Karpathy think about scaling?", even though Karpathy himself has spent the intervening seven years walking that claim back, qualifying it, refining it, and finally largely abandoning it in favor of structured world models. The embedding does not age. The cosine distance does not decay. The system happily retrieves the seven-year-old position because the vocabulary still matches.

This is the context-blindness problem. Standard RAG has no native mechanism for "this was true then, this is true now." You can bolt on metadata filters, sure. You can attempt to rerank by date. Teams do this, and it helps, and it is also a thin tarpaulin draped over a structural hole. The hole is that the retrieval system pushes time back onto you. It asks you to manage temporal validity yourself, badly, with whatever bookkeeping you can hold in your head.

A bi-temporal knowledge graph, by contrast, treats time as a primitive. Every claim has two timestamps: t_event (when the event the claim is about actually happened) and t_ingestion (when the claim entered your system). Every claim has a validity window, a valid_from and a valid_to. When a new claim supersedes an old one, the old claim is not deleted. Its valid_to is closed. It remains in the graph, queryable as historical state, but no longer returned by present-tense queries. Karpathy 2019 and Karpathy 2024 both exist. Both are honored. Neither is averaged into a fictional Karpathy 2021 who never spoke.

This sounds like a small thing, and it is the difference between an agent that says "Karpathy believes scaling is all you need" with full confidence (because it found a 2019 transcript) and an agent that says "Karpathy's 2019 position emphasized scaling; he has since revised this view, and as of 2024 emphasizes structured world models." The first is wrong. The second is right. The difference is bi-temporal modeling, and it is invisible to any system whose substrate is a flat vector cloud.

Failure mode three: attribution loss

By the time a passage of text has been chunked, embedded, retrieved, and concatenated into the model's context, it has been stripped of everything except its words. The provenance is gone. The source credibility is gone. The publication date is gone. The model sees "the optimal learning rate for this architecture is approximately 3e-4" and has no idea whether this came from the foundational paper, a Stack Overflow answer, or a tweet from someone who is angry on the internet. They are all just context tokens now.

This is the attribution-loss problem, and it is the failure mode that quietly destroys most production RAG systems over time. In the early days, when the corpus is small and curated, the sources are all decent and the loss of attribution does not bite. But every corpus grows. Sources get added. Quality drifts. Eventually some piece of low-quality content gets in, matches well on cosine similarity, and the system confidently surfaces it as if it were the foundational paper. There is no defense, because the system cannot tell the difference.

In a metagraph, provenance is structural. A claim lives as a node, linked by SOURCED_FROM edges to the source that asserted it, linked by ASSERTED_AT edges to a timestamp, linked by CREDIBILITY meta-edges to the source's track record. When the agent retrieves the claim, it retrieves the whole structural neighborhood. The credibility weights are visible. The source's history is queryable. The agent can ask, structurally, "how often has this source been right about claims in this domain?" and weight accordingly. The provenance is the same kind of object the claim is, sitting in the same graph and carried by the same structure.

And when the source's credibility shifts, when a researcher's predictions prove repeatedly correct, or repeatedly wrong, the weight propagates. The next time their claim surfaces in a query, the system already knows. This is reinforcement learning at the level of the world model itself, not at the level of weights inside a transformer. It is structural learning, and it accumulates.

Failure mode four: hop blindness

The fourth failure mode is the one that bites hardest in genuinely interesting questions. "Which researchers worked with Hinton at U of T who later went to OpenAI?" No single document contains this answer. Answering it requires composing three facts: who worked with Hinton at U of T, who went to OpenAI, and the intersection. Vector retrieval can find documents about Hinton, documents about OpenAI alumni, and documents about U of T, but the composition is not in the embedding space, because composition is a structural operation, not a similarity operation.

You can sometimes paper over this with multi-step prompting. Ask the model to break the question into sub-queries, retrieve for each, then synthesize. This works occasionally. It is brittle, expensive, and the failure modes are silent. The model decides what sub-queries to issue based on what looks plausible, the retrieval steps each lose information, and the synthesis happens at the end with all the same averaging problems we just discussed. The system can answer the question, sometimes. It cannot be relied upon to answer it.

A knowledge graph answers this kind of question natively, because the graph is the composition. A single Cypher query traverses from the Hinton node along WORKED_WITH edges to other researchers, then along EMPLOYED_BY edges to OpenAI, with a timestamp filter on each. The answer arrives as a structural traversal. It is fast. It is correct. It is auditable. And, critically, it composes cleanly with the other three properties we have been building: temporal filtering, contradiction handling, and source weighting all apply at every step.

This is what we mean when we say the metagraph beats the bicycle. The bicycle (RAG with embeddings) is a perfectly good thing to have for trips around the block. The car (structural reasoning over a temporal, epistemic, provenance-rich metagraph) is what you need when the road actually goes somewhere.

What you give up to get here

Name the trade plainly. Going from RAG to metagraph is not free, and the people who tell you it is are selling something. There are real costs, and pretending otherwise is the kind of mistake that gets sober engineers fired by less-sober ones.

You give up simplicity of ingestion. Chunking a document and embedding the chunks is a one-line script. Extracting entities, resolving them against an ontology, identifying claims, classifying claim types, detecting contradictions with existing graph state, attaching provenance, this is a pipeline. A serious pipeline. It needs NLP infrastructure, it needs ontology management, it needs human review for ambiguous cases, and it needs continuous tending as the corpus grows. The ingestion side of a metagraph is, conservatively, an order of magnitude more engineering than the ingestion side of a vector store.

You give up generality of retrieval. Vector retrieval works on any kind of text. You can throw a PDF, a tweet, and a recipe at it and get something back. Graph retrieval requires that the content has been parsed into the graph's schema first. The schema is your friend, it is what enables the structural reasoning, but it is also a commitment. Adding a new content type means adding new parsers, new entity types, new relations. The system is more powerful and less universal. That is a real trade.

You give up, in some sense, the illusion of zero-cost knowledge. Vector stores let you pretend that just having a pile of documents is the same as having a knowledge base. They are not the same. The pile is the raw material. The knowledge base is what you get after you have done the work of structuring the raw material, and a vector store, structurally, has not done that work. It has only made the pile faster to search. The metagraph approach names the work plainly and pays for it. The work is real, and the value is real, and there is no shortcut.

Why now

Three things have changed in the last eighteen months that make the metagraph approach newly viable as a production strategy rather than a research curiosity. First, large language models have become reliable enough at entity extraction and relation extraction that the ingestion pipeline can be built mostly on top of LLM calls rather than fragile classical NLP. That removes a category of brittleness that used to make knowledge graphs prohibitively expensive to maintain.

Second, the infrastructure has matured. Neo4j with APOC, SurrealDB's multi-model engine, Graphiti's bi-temporal abstractions on top of Neo4j, these are production-grade tools that did not exist, or did not work well enough, three years ago. You can today, with a small team, stand up a temporal knowledge graph with metagraph-style epistemic patterns and have it serve real queries at real latencies. That was not true in 2022.

Third, and most importantly, the agentic frontier has revealed what flat retrieval cannot do. Every team building serious agents, agents that take actions, accumulate memory, reason across multiple turns, handle contested topics, has hit the wall we described above. Some have papered over it with elaborate prompting. Some have given up and shipped narrower products. The teams that are going to win the agentic decade are the ones who recognized the wall is structural and started building the substrate that gets through it. That window is open right now. It will not be open forever.

The shape of the substrate

The substrate is a metagraph: a graph where edges can target edges, where every fact has a truth value, where time is a primitive, and where reasoning is structural rather than statistical. Concretely, in production today, this looks like Graphiti on top of Neo4j for the temporal graph core, with SurrealDB sitting alongside as a unified multi-model layer for vector and full-text retrieval, with a custom contradiction engine running continuously over the graph, with a PLN-inspired confidence propagation system updating truth values as new evidence arrives, and with the whole thing exposed to agents via three primitive operations: query, assert, and update_belief.

Each of those design decisions deserves its own essay, and we will give each of them one over the course of this series. But the principle, the thing that holds the design together, is this: knowledge is structure, not similarity. Two facts can be syntactically identical and epistemically different. They can be cosine-near and structurally opposed. They can look the same to an embedding and behave the same to an embedding and be different facts about different things, said by different people, at different times, with different weights. Only a substrate that holds that difference structurally can reason with it. And only an agent that reasons structurally can be trusted with anything that matters.

What this series will cover

Twelve essays. Each one drills into a specific architectural decision that distinguishes a metagraph-based knowledge system from the RAG-shaped systems most teams are building today. We will go deep on bi-temporal modeling in essay two, on the contradiction engine in essay three, on the metagraph patterns themselves in essay four, on probabilistic truth values in essay five, on the ingestion pipeline in essay six, on the Agent Redwood design framework in essay seven, on Graphiti in production in essay eight, on the maturity of OpenCog Hyperon in essay nine, on hybrid retrieval in essay ten, on episodic memory as a structural moat in essay eleven, and on second-level thinking for infrastructure bets in essay twelve.

The thesis under all twelve is the same. The teams who treat knowledge as structure will produce agents that reason. The teams who treat knowledge as similarity will produce agents that retrieve. Both kinds of agents have markets. Only one of them has a future as the substrate for everything else. The bet we are making with Wiki Design Co. is that the structural kind is the substrate, and that the right time to build it is right now, before everyone else figures out the same thing and the structural moat closes behind us: the years of accumulated episodic data, the careful schema decisions, the contradiction history, all of it compounding.

The next essay starts where this one leaves off: with time. With the question of what it means for a fact to be true at one moment and not at another, and with the bi-temporal model that lets a knowledge system hold that distinction without averaging it into noise.

---

END OF ESSAY 01 · CONTINUE TO ESSAY 02 →