Before You Blame the Model, Check What Your Compactor Deleted
AI agents can look forgetful when context compaction drops critical state. Typed summaries and regression probes make that failure measurable.
Many AI production issues appear to be intelligence failures, such as asking for data users already provided or ignoring stated constraints. The default assumption is that the model hallucinated, but often the real issue is that the context fed to the model was compressed improperly. This article explains how to fix that failure mode using a lightweight probe harness.
Readable summaries can still be operationally wrong. Production systems need to check if the exact state needed for the next decision survived compression. When state drops, downstream behavior can look irrational even if the model is behaving consistently with the compressed input.
In various domains, there are high-impact, low-frequency fields that prose-style compression tends to blur first. These include healthcare contraindications, finance approval thresholds, legal jurisdiction constraints, and supply chain cold-chain requirements.
When investigating a failure, measure the token composition before touching prompts. If tool payloads make up a large portion of the context, compressing dialogue alone is a minor optimization. Clear spent tool payloads before compression to remove stale high-volume noise and reclaim tokens.
Compact the state into a typed structure rather than a narrative summary. Use a TypedDict with fields like entities, constraints, commitments, decisions, open items, and risk flags. This makes missing fields visible, makes output machine-checkable, and makes reviews objective instead of impressionistic.
Add probe questions that are executable questions over the compacted state. For example, check if the scope is still limited to us-east-1, if the "do-not-restart-database" constraint is present, if the error rate threshold is preserved, and if the platform engineer is still the owner of the traffic-shift approval. If these probes fail, the regression is caught before a production decision is made.
Separate the CI pipeline into two lanes. The deterministic lane tests probe predicates and scoring against fixed state fixtures without any network calls or model dependencies. The compactor eval lane runs fixed transcripts through the real compactor and may call the model, running nightly or as a release-quality gate. This keeps developer feedback fast while still catching model/prompt drift.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
