Rohit Ghumare · August 3, 2026 · 8 min read

The LLM Wiki: Stop Retrieving, Start Compounding

Point an LLM at a pile of documents and the usual answer is retrieval. Upload the files, and on every question the model finds the relevant chunks and reasons over them fresh. It works, and it also has amnesia. The subtle question that needs five documents synthesized gets synthesized from scratch every single time, and nothing is ever built up. There is a better shape, and it is old enough to have a name from 1945.

Retrieval forgets; a wiki accumulates

The retrieval pattern, the one behind most document chat, treats your sources as a lookup table. Ask a question, fetch the matching passages, generate an answer, discard everything. The next question starts cold. If two questions need the same cross-document synthesis, the model does that synthesis twice, because the first one left no trace. Knowledge is rediscovered, never retained.

The alternative, laid out cleanly in Andrej Karpathy's LLM Wiki idea file, inverts the flow. Instead of retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki: a structured, interlinked set of markdown pages that sit between you and the sources. When a new source arrives, the model does not just index it. It reads it, extracts what matters, and integrates it, updating entity pages, revising summaries, flagging where the new data contradicts old claims. The knowledge is compiled once and then kept current, not re-derived on every query. The cross-references are already there. The contradictions are already flagged. The synthesis already reflects everything you have read.

Step through both and watch them diverge.

After five sources the retrieval side is five loose documents that the model must re-read in full on every question. The wiki side is a small web of pages that already knows how the sources connect. Ask a question of the first and it works, slowly, from scratch. Ask the second and it reads a handful of already-synthesized pages. The gap only widens with scale.

Three layers

The pattern is deliberately simple. It has three layers, and keeping them separate is most of the discipline.

• Raw sources. Your curated collection of documents, articles, papers, transcripts. Immutable. The LLM reads from them and never writes to them. This is your source of truth.
• The wiki. A directory of LLM-generated markdown: summaries, entity pages, concept pages, an overview, a synthesis. The LLM owns this layer entirely. You read it; the LLM writes it.
• The schema. One document, a CLAUDE.md or AGENTS.md, that tells the LLM how the wiki is structured and what to do when ingesting, answering, or maintaining. This is the configuration that turns a generic chatbot into a disciplined wiki maintainer.

The mental model Karpathy gives is exact: Obsidian is the IDE, the LLM is the programmer, the wiki is the codebase. You browse the pages and follow the links while the model edits based on your conversation. You never write the wiki yourself. You curate sources, explore, and ask the right questions. The LLM does the grunt work: summarizing, cross-referencing, filing, bookkeeping.

Three operations

Everything the wiki does reduces to three verbs.

Ingest. You drop a source in and tell the LLM to process it. It reads, discusses the takeaways, writes a summary page, updates the index, revises the entity and concept pages the source touches, and appends a line to the log. One source might touch ten or fifteen pages. That fan-out is the whole point: the maintenance that a human would skip is the maintenance the LLM does for free.

Query. You ask a question. The LLM finds the relevant pages, reads them, and answers with citations. The insight most people miss is the follow-through: a good answer gets filed back into the wiki as a new page. A comparison you asked for, a connection you found, an analysis, these are worth as much as an ingested source, and they should not vanish into chat history. Your explorations compound alongside your reading.

Lint. Periodically, ask the LLM to health-check the wiki: contradictions between pages, stale claims a newer source has overtaken, orphan pages with no inbound links, concepts mentioned everywhere but lacking a page, missing cross-references. The model is good at proposing the next question to investigate and the next source to find. This is what keeps a growing wiki from rotting.

Two files hold it together. index.md is the content catalog, every page with a one-line summary, read first on every query to find what is relevant. log.md is the append-only timeline of ingests, queries, and lint passes. Give each log line a consistent prefix and it stays greppable: grep "^## \[" log.md | tail -5 is your recent history. At the scale of a few hundred pages, the index file alone does the job of search, no embedding infrastructure required.

The schema is the real product

It is worth saying plainly, because it is the part people underweight: the schema document is the most important file in the system. It encodes what entity and relationship types exist in your domain, how to ingest each kind of source, when to make a new page versus update an existing one, what quality bar to hold, how to resolve contradictions. You and the LLM co-evolve it. The first version is rough; after a few dozen sources and a few lint passes it reflects how your domain actually works, and at that point it is transferable. Hand it to someone working in the same domain and they get a running start. The wiki is the output. The schema is the program that produces it.

An old idea, finally cheap

In 1945 Vannevar Bush described the Memex: a personal, curated store of knowledge with associative trails linking one document to the next, the connections as valuable as the documents. It was closer to this than to what the web became: private, actively maintained, associative. Bush could not solve the one hard part, which is who does the maintenance. Updating cross-references, keeping summaries current, noting every contradiction, holding consistency across dozens of pages: this is the work humans abandon, and it is exactly the work that kills wikis. The maintenance burden grows faster than the value, so people stop.

The LLM removes that constraint. It does not get bored, does not forget to update a back-reference, and touches fifteen files in one pass. The wiki stays maintained because maintenance now costs almost nothing. That is the whole unlock: not that the model is smart, but that the bookkeeping is finally free. Your job is to curate, direct, and ask. The model's job is everything else.

That is the pattern at its cleanest. The interesting question is what breaks when you run it hard, across thousands of sessions, past a few hundred pages, with more than one agent writing. That is the subject of the companion piece, LLM Wiki v2, on the lifecycle, graph, and search machinery the simple version leaves out.