Context Engineering for Agentic Workflows: Task-Aware Token Budget Forecasting

The author's initial observation was not about the accuracy of an agent's responses, but rather their concern for the shape of the token curve during an agent session. The agent was sending excessive historical data, tool outputs, and instructions to the model, which were no longer needed. This waste of token budget was consistent across agents, leading the author to conclude that the issue lay with the lack of context management.

The author proposed moving context management to a separate layer beneath the agent, rather than within the agent's logic. This layer would forecast the token budget per task before the agent runs, rather than discovering the cost afterward. The author emphasized that agent tasks fall into a few classes, and within a class, the context needed is fairly stable.

By forecasting the token budget for each class, the platform could better manage costs. The framework consists of four stages: selection, reranking, clustering, and finally, context compression. Selection determines the most relevant candidates for the task, while reranking scores each piece based on its relevance to the current step.

Clustering groups similar candidates together, and context compression fits the content into the predetermined caps. The author found that this approach not only reduced costs but also helped identify agents that were unnecessarily hoarding context.