Types of Context Rot in Harness Engineering

Types of Context Rot in Harness Engineering

Context rot is the name for the degradation of an LLM-powered agent's reasoning as it processes increasingly large amounts of context. It is not a single problem, but five distinct failure modes that combine and worsen as the context window grows. Understanding these five failure modes is crucial for building robust harnesses that maintain high reasoning quality even with extensive context.

1. Lost in the Middle: Models tend to pay more attention to content at the beginning and end of the context window than to content in the middle. As sessions grow, important content gets pushed into the middle where it is increasingly ignored. System prompts and latest user messages always get strong attention, while middle content gets progressively less attention.

2. Attention Dilution: Transformer attention is a fixed resource that gets spread thinner as more tokens are present. An instruction that worked perfectly in a short context may be effectively ignored in a long context due to competition for attention with more tokens. System prompts working beautifully in fresh sessions may feel like they "lose their grip" after several tool calls and retrieved chunk dumps have crowded into the context.

3. Distractor Amplification: Irrelevant content in context doesn't just fail to help, it actively hurts. The model reasons about all content present, including irrelevant information. If a search returns multiple sources with one off-topic snippet, the model may incorporate the off-topic snippet into its reasoning, producing confident and coherent but completely wrong answers.

4. Repetition Bias: Content that appears multiple times in context gets weighted more heavily by the model. This bias shows up when the same instruction or tool result is repeated multiple times, making the model more likely to follow that repeated content. Repetition bias compounds as more irrelevant or off-topic content is retrieved and added to the context over the course of a session.

5. Session Context Confusion: As a session grows, the distinction between system prompt, user messages, and tool results blurs. The model may confuse which content belongs to which role, leading to reasoning that mixes system instructions with user input or tool outputs. This confusion can result in the model following inappropriate instructions or using outdated tool results.

These five failure modes interact and compound over time, making context rot a complex problem to address. However, by understanding the distinct mechanisms at play, harness engineers can implement targeted solutions to mitigate each failure mode. Techniques such as prioritizing system prompts and latest user messages, limiting attention dilution through pruning irrelevant content, filtering out distractors, avoiding repetition bias with unique content, and carefully managing session context are all essential steps in building robust, high-quality agentic harnesses.