CoReMem: Cost-Internalizing Memory for Successive Coding Tasks with Prospective Valuation

CoReMem is a memory system designed to help coding agents manage memory costs and values when working on a series of coding tasks on an evolving codebase. The system treats memory selection as a cost-benefit problem, estimating the prospective value of each memory segment by comparing outcomes when the segment is included versus excluded during the next task.

CoReMem runs the next task twice for each candidate segment: once with the segment included in memory, and once without. The difference in resolution rate and step count between the two runs determines the segment's value. Segments that improve resolution or reduce step count are considered valuable, while those that don't improve or slow down the agent are considered worthless.

The system organizes memory around two types of anchors: location anchors (file paths, function names, class definitions, and variable scopes) and executable anchors (test commands, linter invocations, build steps, and runtime checks). Each anchor links to the trajectory segment that produced it. When the agent needs to retrieve a segment, it does so by accessing the associated anchor.

The agent learns to act on memory or wait for more context based on a reward function that prioritizes resolution and penalizes step count in successful runs. The reward is calculated as resolution bonus minus the step count multiplied by a step penalty. The agent runs rollouts with and without memory access, comparing on-policy reward estimates to determine the memory's value.

Memory value can become negative in three scenarios: stale anchors, noisy anchors, and suboptimal anchors. Stale anchors refer to outdated references to code elements that have changed. Noisy anchors are segments that provide little value or introduce errors. Suboptimal anchors are segments that provide some value but are not the most efficient choice.

To mitigate these issues, CoReMem uses techniques such as aggressive pruning of low-value anchors, conservative pruning based on anchor usage frequency and recency, lightweight provenance tracking to minimize instrumentation overhead, and careful tuning of the reward shaping parameters, including the step penalty coefficient and exploration budget.