AI Agent Memory: What It Is and How It Works

The taxonomy that has settled in agent work borrows from cognitive science and holds up well in practice. Working memory is the state of the task in progress. Episodic memory is the record of what happened in a specific past run. Semantic memory is the set of facts that are true regardless of which run you are in. Procedural memory is how a recurring job is carried out.

Chat products care most about the first two. They want to know what you said earlier in this conversation, and what you told them three weeks ago. That is why so much memory tooling is shaped around a user and a conversation: those products were the first to need it, and they defined the category's default assumptions.

A coding agent has the opposite weighting. Its working memory is handled by the harness, and its episodic memory is mostly noise, because the interesting content of last Tuesday's session is not the sequence of events but the one conclusion that came out of it. What it actually needs is semantic and procedural: the decisions and constraints that govern this repository, and the way this team does recurring work. That is a different retrieval problem, and reaching for a tool built for the first pair is the most common category error in this space.

Every memory system has a write path with four stages, whether or not it names them. First, extraction: something has to decide that a piece of text is worth keeping. This is either automatic, where a model reads a transcript and proposes facts, or explicit, where code or a person writes the entry deliberately. Automatic extraction scales and produces a lot of confidently phrased trivia. Explicit writes are sparse and much higher signal.

Second, deduplication. Without it, the same fact accumulates in fifteen slightly different phrasings, and every subsequent retrieval spends its budget on variations of one idea. Third, conflict resolution, which is where the designs genuinely diverge. The simplest systems overwrite. Some append and let the retrieval layer sort it out, which quietly guarantees the agent will eventually be handed two contradictory facts with no way to rank them. The more careful ones close the previous fact with a validity window so the history stays queryable, which is what a temporal knowledge graph buys you.

Fourth, and most often skipped, a policy for keeping the store honest. Team knowledge goes stale on a schedule nobody tracks. A memory layer with no review surface and no decay is a system for making last quarter's convention permanent, and an agent that confidently applies an outdated rule is worse than one that knows nothing, because the output looks informed.

The read path has two decisions, and people usually only discuss the first. Selection is how the system narrows a store to the subset worth sending. Embedding similarity is the default: turn the query and the stored entries into vectors and return the nearest ones. It is good at paraphrase and bad at precision, which is why keyword ranking such as BM25 is often blended in. Graph traversal follows relationships rather than measuring distance, and is the right shape when the question is about how entities connect. Scope matching does no ranking at all: it returns everything attached to an identifier, such as a directory, and works when the identifier is a genuinely good proxy for relevance.

Injection is the second decision, and it determines more about the felt behaviour than the first one does. Memory can be pasted into the system prompt at the start, which is simple and puts a fixed cost on every run. It can be exposed as a tool the agent chooses to call, which is efficient and defers the decision to the model. Or it can be pushed at a hook point, before the agent takes its first action, which costs a runtime on the machine and removes the model's discretion.

These two decisions are close to independent, and you can reason about any product on this grid. A tool with excellent selection and tool-call injection behaves very differently from one with crude selection that always fires. Feature lists rarely make the distinction, which is why comparing them tends to produce a tie.

Tool-call injection has a specific failure that is worth understanding on its own, because it is the single most important property of the read path and it is almost never listed as a limitation.

An agent queries a memory tool about things it knows to wonder about. It does not query about a convention it has never heard of, because nothing in the prompt raises the question. So the knowledge most worth having, the constraint that would have prevented the mistake, is exactly the knowledge least likely to be retrieved. There is no error, no empty result and no signal of any kind. The agent writes something plausible and wrong, and the memory that would have stopped it sits in the store, correctly indexed, never asked for.

This is not an argument that push beats pull. Pushing costs something real: it needs a process on the machine that can see the working directory and participate in the agent's startup, which a hosted API cannot do, and it spends context on every run whether or not the run needed it. The honest framing is that pull optimises for cost and push optimises for coverage, and a constraint that must never be violated is a coverage problem.

There are four realistic homes, and they trade off along the same two axes every time: who else can see it, and when it arrives. The context window is free and instant and dies with the session. A file checked into the repository, such as CLAUDE.md or AGENTS.md, survives, is reviewable in a diff, and travels at commit speed, which means a teammate does not have your knowledge until they pull. A local store, which is what the reference MCP memory server gives you, survives and needs no account, but is invisible to everyone else and to your other machine.

The fourth is a service outside every agent. That is the only one of the four that lets knowledge written by one person reach a teammate's session without a commit, and the only one where the same entries can drive two different agents. It is also the only one that adds an operational dependency, which is a real cost and not a detail.

The reason to be explicit about this is that most teams reach for the fourth option while their actual problem is still solved by the second. The break point is not a feature you are missing. It is the arrival of a second agent or a second person, whichever comes first.

The silent miss is the one above: the agent never asked, so it never received, and nothing anywhere reports a failure. It is the most expensive failure because it is indistinguishable from the system working.

Staleness is the second. A fact enters the store when it is true and stays after it stops being true, and because retrieval has no notion of freshness, the agent applies it with the same confidence as everything else. The fix is procedural rather than technical: someone has to be able to see what is stored and correct it, which means the store needs to be legible to humans and not only to an index.

The third is unbounded growth. Automatic extraction with no deduplication produces a store where every retrieval returns near-duplicates of the same three ideas, and the effective recall of the system falls as its size rises. The fourth is the wrong unit, which is the design error that no amount of tuning recovers from: keying knowledge to a user when the thing it describes is a directory, or to a directory when the thing it describes is a person. The unit is the first decision and the hardest to change later.

Pathrule is one concrete point on the grid above, and it is worth walking through as an example because it makes the two read-path decisions in an unusual combination. Selection is scope matching rather than similarity: entries are attached to repository paths, and the slice for /services/billing is whatever is attached at or above that path. Injection happens at the hook points the coding agent already exposes, before the first tool call, so delivery does not depend on the agent deciding to ask.

The write side is deliberately typed rather than uniform. Memories hold what is true, rules hold what must not happen, and skills hold how a recurring job is done, which maps onto the semantic and procedural split from earlier. Typing them separately is what lets a rule carry a priority and an enforcement level, so a strict rule can block a matching change instead of merely describing why it should not happen. Prose in a single instruction file cannot make that distinction, because a Markdown bullet has no way to say which of its sentences is load bearing.

The trade is stated plainly in the section above and applies here too. This design spends context on every run and needs a local runtime, and it is the wrong choice entirely if your unit is a user rather than a path. The mechanics are documented in how retrieval works, and the product-by-product version of this grid is in best AI agent memory tools for coding.

Write the instruction file first and put it where it is scoped. Both Claude Code and Codex read nested files and concatenate them by directory, so an entry that is only true of one service belongs in that service's directory rather than the root. This costs nothing, is reviewable in a pull request, and solves a surprising fraction of the problem. Most teams have never split their root file, and doing it usually shortens it enough to make it worth reading again.

Then try the reference MCP memory server. It stores entities, relations and observations in a JSONL file on your machine, has no account and no service behind it, and is an honest implementation of the read and write paths described here. For one developer on one machine it is often the whole answer, and finding out how much of your problem it covers is a better first step than evaluating a platform.

Escalate only when the remaining problem has changed shape: knowledge that has to reach a teammate faster than a commit, a constraint that has to hold rather than be suggested, or a second agent that would otherwise need its own copy of everything. Those three are what an external layer is actually for. Everything before them is scoping, and scoping is free.