Agent Memory Is an Architecture Decision: From MCP Servers to Stateful Agents

When developing memory layers for agents, it is crucial to separate three distinct jobs: storing information, retrieving it, and deciding whether to use it at all. Memory tools differ in where they draw these lines, so they should be treated as architectural choices rather than drop-in replacements.

When deciding on the integration boundary, developers must choose whether the application calls memory at fixed points or exposes operations as tools the agent can use. This choice influences whether the runtime is retained and whether answers require extracted facts, source material, or synthesized conclusions. MCP is an interface, not an architecture, and different systems like Hindsight, Mem0, Graphiti, Letta, and Cognee provide varying degrees of control over memory storage and retrieval.

Hindsight, an open-source agent-memory system, makes a clear API distinction between retain, recall, and reflect processes. Recall deals with direct evidence retrieval, while reflect involves reasoning and synthesis over the retrieved material. This distinction helps developers control the control flow and choose appropriate testing strategies.

For a coding assistant, storing decisions, reasons, and project scope together allows for effective testing of new-session questions. However, running reflection on every request should be avoided, as it adds unnecessary processing stages.

Mem0 focuses on memory extraction from messages and scoping through identifiers like user_id, agent_id, and run_id. This approach is suitable for existing applications with orchestration in place, as it enables memory to enter at a specific point in the request lifecycle. Retrieval configuration, such as scope enforcement and the number of returned items, should be tested to ensure the implementation meets the application's expectations.

Graphiti, a temporal graph engine, provides a framework for tracking changing facts, retaining links to source episodes, and retrieving information through semantic, keyword, and graph-based methods. It also offers an MCP server, making it a useful comparison for projects where relationships and time are crucial. Graphiti allows for fine-grained control over the graph-based implementation, enabling developers to differentiate between historical and current escalation contacts.

Letta introduces MemFS, a git-backed memory filesystem, as part of its stateful-agent approach. Unlike adding a service behind an existing request handler, Letta's memory management becomes an integral part of the runtime design. This is particularly relevant for persistent developer assistants, where the agent's ongoing identity and memory management are critical. Evaluating the broader harness and operational lifecycle, rather than just the memory file, is essential for assessing Letta's suitability.

Cognee offers a configurable graph-oriented memory pipeline that separates relational source metadata, vector representations, and entity relationships. By moving session material through a learning pipeline into permanent knowledge, Cognee is particularly valuable for questions spanning code, documentation, and operational decisions.

Its documentation emphasizes the importance of evaluating the connections between different data sources against known evidence. However, the configurable pipeline creates additional choices regarding extraction, backends, and retrieval, requiring careful measurement of ingestion and readiness separately from query latency.