I Gave Claude Code a Brain. This Time I Let the Tests Judge It.
The first experiment measured whether Claude Code knew the right answer. This one measured whether the work actually succeeded. In my previous experiment , I tested whether a memory layer could stop Claude Code from confidently repeating project-specific mistakes. It did. But there was an obvious limitation. The tasks measured whether the answer was correct. They did not ask the agent to modify a…
The recent experiment tested whether Claude Code could complete tasks involving repository modifications and deterministic checkers. The tests compared three configurations: bare Claude Code, Claude Code with a project-specific CLAUDE.md file, and Claude Code with RE-call memory. RE-call significantly improved success rates over the CLAUDE.md baseline, winning 17 out of 24 comparisons.
Memory retrieval was successful in 83.3% of eligible sessions, with useful context retrieved 85.0% of the time. While memory improved the final score, the memory layer cost more in terms of API usage. The experiment demonstrates that a production memory layer can improve task completion accuracy, but further optimization is needed for retrieval efficiency.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
