How Much Memory Does Your Agent Actually Need?

When equipping an agent with agentic memory, the amount of memory needed is not a simple on/off switch, but rather a calibrated dose tailored to the model's capabilities. ALTK-Evolve allows an agent to learn from its own past experiences and distill reusable guidelines, which can then be injected back into the agent's inference at inference time without any weight updates or human annotation.

The key takeaway is that the right dose of memory varies depending on the model tier. Strong models with headroom can benefit from the full guideline set, while weaker models perform best with a compact core plus per-task retrieval. Saturated models, which are already near their performance ceiling, show no measurable gain from additional memory.

Curated retrieval has been found to be both the most accurate and the cheapest option. For example, GPT-OSS-120B gained +16.1pp in task completion using curated retrieval, while the full guideline set only provided a slightly smaller gain (~5% tokens) and cost 50% more tokens. This cost efficiency makes curated retrieval a practical choice for production environments.

The amount of memory a model should use depends on several factors beyond just parameter count, such as benchmark headroom, context window size, architecture, guideline quality, and task distribution. Identifying the optimal memory configuration is an ongoing area of research. However, the practical takeaway is that the right dose of memory can be calibrated based on the specific model being used.