How do you know your agent's memory layer is helping, not hurting?
I work on long-term memory for agents, and the honest problem I keep hitting: everyone benchmarks the model, almost nobody benchmarks the memory layer itself. A bad memory layer can actively make a good model worse, stale facts retrieved with high confidence, contradictions never surfaced. The vendor answer is usually "trust our evals." I published an open harness instead, MIT, that reruns 500…
We haven't written up this one. Dev.to has the full story — the link below goes straight to it.
