{
  "id": 6902085,
  "title": "Agent memory and questions to ask",
  "url": "https://urgent.news/2026/09/12/agent-memory-and-questions-to-ask",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-12T09:10:29.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/vasilije_markovic__/agent-memory-and-questions-to-ask-2jo8"
  },
  "original_language": "en",
  "account": "To evaluate an agent's memory, start by testing its ability to recall specific instructions based on historical notes. Create two fictional projects, Atlas and Beacon, and set up test environments for each. For Atlas, record the initial fact that tests run with npm test. Then, introduce a correction stating that Atlas now uses pnpm for testing. Repeat the same process for Beacon, noting that its test command remains unchanged.\n\nBegin with a fresh session and ask how to run tests for Atlas on September 12. The correct answer should be pnpm test, reflecting the latest instruction. Then inquire about how Atlas tests were run on September 1, expecting the response to be npm test, identified as the historical command. For Beacon, ask how to run its tests on September 12, expecting the answer to still be npm test. Lastly, ask why Atlas migrated to pnpm, and verify that it correctly identifies the September 10 maintainer note as the source of the current command.\n\nBreak down your evaluation into key aspects: retaining corrections, preserving historical information, distinguishing between different projects, recognizing when information is missing, and tracing answers back to their sources. If the agent fails to provide the correct response at any point, investigate the specific stage where the memory system faltered. For instance, if it returns the September 1 note instead of the corrected one, examine the retrieval path and the logic used to rank and select answers.\n\nRepeat the process by importing an old Atlas README dated September 1, after applying the correction. The agent should still return pnpm test, demonstrating that the retrieval process is not influenced by the order in which the notes were stored. This exercise helps you determine whether your memory system can accurately maintain and retrieve different versions of instructions over time.\n\nConsider how your specific memory implementation—whether it uses a graph, vector, relational database, or a knowledge graph—stores corrections and retrieves the appropriate version of information. Each system has its own method for representing changes, such as attaching a supersession relationship to notes or maintaining separate current-state records. Choose the architecture that best fits your application's needs, ensuring it can answer questions about both the present and the past accurately.",
  "summary": "Tell a coding agent that your project runs tests with npm test . Start a new session and ask it how to run the tests. It answers correctly. Good first result. Now tell it the project has migrated to pnpm. Start another session. Ask the same question. Does it give you the new command? Does it offer both? Can it explain which instruction is current? That second conversation is where I'd start…",
  "key_points": [
    "Evaluate agent memory by testing recall of specific instructions based on historical notes.",
    "Agent should correctly identify September 10 maintainer note as source of current command for Atlas."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}