A Memory Architecture for AI Agents
An agent that remembers everything in one big file pays for all of it on every request and still struggles to find the relevant bit. The fix is layers: load what's always needed, index the rest, and pull detail on demand.
The Core Principle
Not all memory should load the same way. Stable rules load every session. A lean index loads every session but only points at detail. The detail itself loads only when relevant. Separating these keeps the always-on cost low while keeping deep knowledge reachable.
The Layers
| Layer | Loads | Use for |
|---|---|---|
| Rules / conventions | Fully, every session | Stable "always / never" instructions |
| Index | Every session, capped | One-line pointers to topic files |
| Topic files | On demand | Detailed knowledge, one topic each |
| Project state | Per project | Context, contacts, status for one project |
The Rules That Keep It Working
The index is an index, not content
The always-loaded index should hold one-line pointers to topic files, not the knowledge itself. Multi-line blocks belong in topic files. Every line of content you jam into the index spends the budget that should point at ten other things.
Cap the index and leave headroom
There's a practical ceiling on how much loads every session. Target well under it. Many systems truncate silently past the cap — you won't get a warning, the agent will just quietly stop seeing the newest entries. Put high-value pointers near the top.
Descriptions drive retrieval
Each topic file needs a clear description in its header. Agents typically scan metadata first, then load a handful of relevant files. Vague descriptions mean the right file never gets pulled. Write the description for the future search, not for yourself today.
One topic per file, kept short
Keep topic files focused and small. A file that sprawls across three subjects is one the agent loads for the wrong reason and then can't use efficiently. Split by domain; link related files rather than merging them.
Derive from source, don’t duplicate it
Don't store what the code, the file tree, or version history already records. Stale memory is worse than no memory: the agent trusts a remembered path or value that has since changed. If it's derivable from live state, let it be derived.
Common Mistakes
Don't: Content in the index instead of topic files
Do: Index holds one-line pointers only
Don't: Rules living in the memory index
Do: Instructions go in the always-loaded rules layer
Don't: No rotation plan (silent truncation)
Do: Cap the index, keep headroom, prune regularly
Don't: Trusting stale memory over live state
Do: Verify paths/values against the source before acting
Don't: Orphaned topic files with no pointer
Do: Every topic file gets an index entry, or it’s never found
Don't: Same fact duplicated across layers
Do: One home per fact