Persistent Memory for AI Coding Agents: Six Decisions

The baseline is not nothing. Both Claude Code and Codex read nested instruction files and concatenate them by directory, root first, so a nearer file lands later and wins. That means a fact that is only true of one service belongs in that service's directory rather than in the root file, and most teams have never split their root file. Doing it usually shortens the root enough to make it worth reading again, and it costs nothing.

A file has three genuine properties a hosted layer struggles to match. It is reviewable: a change to a convention arrives as a diff with an author and a reason, in the same pull request as the code it governs. It is portable: no account, no runtime, no export question. And it is versioned with the code, so checking out a six month old branch gives you the conventions from six months ago, which is occasionally exactly what you want.

It breaks in three places, and they are worth being precise about because they are the only honest reasons to escalate. It travels at commit speed, so a teammate does not have your knowledge until they pull. It has no types, so a Markdown bullet cannot say which of its sentences is load bearing and which is a suggestion. And it grows: a single file that every scope loads gets longer, and the longer it gets the lower the share of it the model actually follows. Two long reads on those failure modes are why Claude Code ignores your CLAUDE.md and managing AI coding context in a monorepo.

The practical rule: the file is the baseline every other option has to beat, and you cannot tell whether an option beats it until the file has been split. Buying a layer to fix a root file nobody has scoped is paying monthly for a folder structure.

This is the first decision and the one that does not survive being wrong. Every memory system attaches an entry to something, and the something determines what retrieval can even express. The common units are a user, a conversation, a document, an entity, and a location in a repository.

Most of the tooling in this space attaches to a user, because chat products needed memory first and defined the category's defaults. That is the correct unit for knowledge about a person: their preferences, their history, what they told you three weeks ago. It is the wrong unit for a coding agent, because the fact that a discount attaches to the line item rather than the order total is not true of you. It is true of services/billing, and it stays true when you leave the company.

The entity unit is what a knowledge graph gives you: a node called OrdersService with observations attached and relations to other nodes. It is expressive, and it is the model behind the Model Context Protocol's own reference memory server. Its cost is an extraction step that has to decide what counts as an entity, and that step is where most graph memory systems quietly become inconsistent.

The path unit is the one that matches how a repository is organised, and it has an unusual property: it needs no ranking at all. If an entry is attached to services/billing and the agent is working in services/billing, the entry is relevant by construction. That is perfect precision when the path is a good proxy for the subject, and useless when it is not, which is why it works well for repository conventions and badly for anything about a person.

There are four selection methods in production use, and the honest summary is that each is excellent at exactly one thing and mediocre at the rest.

Embedding similarity turns the query and every stored entry into vectors and returns the nearest ones. It is the default because it handles paraphrase: a prompt that shares no words with the entry can still match it, including across languages. Its weakness is precision. A query that mentions a specific file name gets no special treatment, because a file name is just another token in the vector, and the entry that mentions it competes on equal terms with five entries about the same general topic.

Keyword ranking, usually BM25, is the mirror image. An exact identifier, a function name, a literal phrase from a title: these it nails, cheaply, with no model and no embedding store to maintain. It misses the memory that says the same thing in other words, which for a team knowledge base is most of them, because the person who wrote the memory and the person prompting the agent are rarely using the same vocabulary.

Graph traversal does not measure distance, it follows relationships. This is the right shape when the question is about how things connect rather than what resembles what, and a temporal graph adds something the other methods cannot express at all: a fact can be closed with a validity window instead of overwritten, so the history stays queryable and the agent can be told that something used to be true. The costs are real. You need an extraction step that builds the graph, and the graph implementations in this space generally want a graph database behind them, which is a service to run rather than a file to write. Related read: see your team's context as a knowledge graph.

Scope matching does no ranking. It returns whatever is attached to an identifier, most usefully a directory. Nothing is scored, nothing is approximate, and the candidate set is small before anything expensive runs. It is the cheapest of the four and it is entirely dependent on the unit decision above being right.

In practice nobody ships one of these alone, and you should be suspicious of a tool that claims to. Blending keyword and embedding is standard, because an exact match keeps its priority while a same-idea-different-words match still surfaces. Narrowing by scope before either one is what keeps the blend affordable, since the expensive comparison then runs over a handful of candidates rather than the whole store.

There are three homes, not two, and the middle one is the one people skip. A local store lives on your disk: the reference MCP memory server writes JSONL, and other local-first tools write plain Markdown you can open in an editor. No account, works offline, nothing crosses a network. It is invisible to your teammate and to your other machine, which is the entire limitation and it is a large one.

The middle option is a file inside the repository. It is technically local and effectively shared, because git distributes it. It arrives at commit speed rather than instantly, and it is the only option where a knowledge change goes through code review, which is a property worth more than it usually gets credit for.

A hosted service is the only home where knowledge written by one person reaches a teammate's session without a commit, and the only one where two different agents on two different machines read the same store. It is also the only one that adds an operational dependency, a data boundary question your security review will ask about, and a bill. Those are real costs, not footnotes, and a team whose actual problem is an unsplit root file should not be paying them yet.

One question belongs in this decision and is almost never asked: what happens to the store if the vendor stops shipping it. This is not hypothetical. The local-first server most people in this category had bookmarked, Mem0's OpenMemory, has quietly stopped shipping: its directory is gone from the Mem0 repository and the documentation URL for it now lands on the docs home. Anyone who standardised a team on it is doing a migration nobody scheduled. Ask for the export format before you need it. A store you can read as plain files has already answered the question.

This is the decision that determines how the system feels, and it is the one least likely to appear on a comparison page.

Pull means the agent calls a tool when it decides it needs something. It is efficient, it defers the judgment to the model, and it is the only thing a cloud-only connector can do, because a hosted endpoint cannot see your working directory or take part in the agent's startup. For a lookup the agent knows it needs, pull is exactly right.

Pull has one structural failure, and the reason it belongs in a decision list rather than a footnote is that retrieval quality neither causes it nor can fix it. What fails is a query that was never issued. Whatever prompts a lookup has to already be present in the turn, and a convention nobody has mentioned prompts nothing, so an entry's value to the team and its odds of being fetched end up moving in opposite directions. Worse, the miss is unobservable from inside the system: no exception, no empty result set, no line in a log. The tool's own metrics describe a healthy layer for the entire session in which it changed nothing. The read-path version of this argument, drawn out properly, is in AI agent memory: what it is and how it works.

Push closes that gap by injecting at a hook point before the agent's first action. Claude Code exposes four events for this and they differ in what they can do: SessionStart adds context once per session and cannot block, UserPromptSubmit adds context per turn and can block, PreToolUse can inspect a tool call, add context, and deny it, and PostToolUse can add context after the fact but cannot block, because the tool already ran. That last distinction is what separates a rule that describes a constraint from a rule that enforces one.

Push costs something real and the honest framing matters. It needs a process on the machine that can read the working directory and participate in startup, which rules out a purely hosted setup. It spends context on every run whether or not the run needed it. OpenCode's own documentation puts the general version of that warning plainly: MCP servers add to your context, so be careful which ones you enable. The trade is a budget question set against a coverage question. Pulling spends nothing until the agent asks, and therefore reaches only as far as the agent thought to reach. Pushing spends on every turn and reaches what the turn did not know it wanted. If the errors that actually hurt you are lookups the agent knew it needed, pushing is optimising the wrong axis and paying for the privilege. The server-by-server version of this argument is in memory MCP servers for AI coding agents.

Most teams adopt a memory layer, feel that sessions got better, and never measure anything. That is a defensible way to run a small team and a bad way to decide whether to keep paying. Three metrics cover it, and the first one is the only one that catches the failure mode that matters.

Coverage. Did the constraint that should have arrived actually arrive. Construct the test the way the failure happens: take a task whose correct answer depends on a convention that is not stated anywhere in the source, run it with the layer and without, and check whether the agent gets it right on the first pass. This is the only measurement that detects a silent miss, because every other metric looks fine when the agent confidently does the wrong thing.

Footprint. Tokens the model processes per turn. This is what path scoping and selective delivery are supposed to reduce, and it is easy to measure honestly. Report billable non-cached tokens alongside the total, because prompt caching flatters a large static instruction file and will make a dump look cheaper than it is.

Correctness under the smaller context. A layer that cuts tokens and also cuts accuracy has not helped. Score mechanically against expected facts and required actions rather than by reading transcripts, or you will grade your own homework.

Two things make a published benchmark trustworthy, and their absence is diagnostic. The first is a pinned, real repository rather than a synthetic fixture. The second is that the report contains cells where the tool being sold loses. A benchmark with no losing cells was not run to find out.

Teams rarely stay on one agent, and the moment a second one shows up the memory question splits in two. The tool half is mostly solved. The knowledge half is not.

The tool half: MCP means a memory server is written once and any client can call it, which is a genuine improvement over one integration per tool per client. What is not standardised is where each client keeps its configuration. Claude Code takes claude mcp add with three scopes, writing to ~/.claude.json for local and user scope and to a committed .mcp.json at the project root for project scope, and it resolves conflicts local first, then project, then user. Codex reads TOML from ~/.codex/config.toml under [mcp_servers.<name>], and its codex mcp add only handles stdio servers, so an HTTP server has to go into the file directly. Cursor uses .cursor/mcp.json. OpenCode uses an mcp object in opencode.json with an explicit type of local or remote. Same server, four formats, four places to forget to update. Long read: connect an MCP tool once for every AI engine.

The knowledge half is the harder one and MCP does not touch it. Claude Code reads CLAUDE.md, Codex reads AGENTS.md, Cursor reads its own rules directory, and each of those is a separate copy of the same conventions with a separate drift path. Two of the three will be out of date within a quarter, and nothing will tell you which two. The only structural fix is to hold the knowledge in one place that every agent reads from, rather than in one file per agent. That is the argument in one knowledge layer for Claude Code, Cursor and Codex, and the reason team memory should not change when AI tools do.

Pathrule is one point on this grid rather than the only sensible one, and it is worth walking through as a worked example because it makes an unusual combination of choices.

Unit: a repository path. Memories, rules and skills attach to a node path, and the slice for /services/billing is what is attached at or above that path. Selection: scope first, then a blend. The path narrows the candidate set before anything expensive runs, then keyword and meaning-based matching run together over what is left. Timing: push by default, pull when the hook is not enough. The matching slice arrives at the hook points the agent already exposes, before the first tool call, and the MCP tools are there for the deep path: discovery prompts, full bodies, and writes. Location: hosted, with a local runtime. The runtime on the machine is what makes hook delivery and offline reads possible, and it is also the boundary that keeps source code local, since no Pathrule tool reads your files.

Two of the choices are typed rather than uniform, which is the part that a flat file cannot copy. Memories hold what is true, rules hold what must not happen, and skills hold how a recurring job is done. Because a rule is its own type it can carry a priority and an enforcement level, so a strict rule can block a matching change at PreToolUse rather than merely describing why it should not happen. And because a skill is a procedure rather than a fact, it is offered by name and never pushed as a body: handing an agent a checklist it did not ask for instructs it rather than informs it. The mechanics are in how retrieval works and what arrives in full, and what arrives as a name.

Measurement, kept in two separate sets that must not be merged. The first is a single knowledge-heavy reference task on a real codebase, run with and without Pathrule: about 85% fewer input tokens, 5 to 8 times faster, about 5 times fewer tool calls, about 80% lower cost. The second is an open benchmark you can run yourself against a pinned checkout of Fastify, scored mechanically, where the hard tier shows about 52% lower total footprint on Claude with facts and actions unchanged and about 41% lower on Codex with 1.6 points lower fact accuracy and 33.3 points more required actions followed. That Codex row is a mixed result and it is published as one. The conditions for both are in how we measure.

Where this is the wrong answer: if your unit is a person rather than a path, a user-keyed memory service is the correct tool and Pathrule is not. If you are one developer on one machine, the reference MCP memory server or a local Markdown store is smaller and better. If you cannot run a local runtime, the hosted endpoint works but delivers on request rather than before the first tool call, which gives up the coverage property that is most of the argument.

Answer the unit first. It constrains everything after it and it is the only one of the six you cannot change later without rewriting the store. If the knowledge you keep losing describes a place in the repository, do not adopt a tool built around a user id, however good its recall is.

Answer timing second, because it decides whether a whole class of tool is even a candidate. If the failures that hurt are constraints the agent never thought to ask about, you need something that pushes, and pushing needs a process on your machine. If the failures are lookups the agent knows it wants, pull is cheaper and a hosted connector is enough.

Selection and storage location come third and fourth, and they are the two most people start with. They matter, but they are tuning decisions on top of the first two, and a good answer to either cannot rescue a wrong unit or the wrong arrival time.

Then do the free work and wait two weeks before buying anything. Push what is only true of one service down into that service's directory, delete the lines that describe a convention the team abandoned, and read what is left. The remainder is your actual brief, and it is normally smaller, sharper and less like a product category than the problem you set out with. Three things survive that exercise: knowledge that has to reach a teammate faster than a commit can carry it, a constraint whose violation is a regression rather than a style disagreement, and a second agent that would otherwise need its own copy of everything. If none of the three is on your list, the six decisions above are interesting and you do not have to make them yet.