Your Chat History Is Eating the Context Window — Fix It With a Reducer

Every conversation with an AI chatbot can quickly become unwieldy. A support chat may handle nine turns without issue, but a conversation stretching to ninety turns introduces challenges. Each request brings a system prompt, three agent loops of tool noise, repeated explanations of the same issue, and tangents from the past. The AI model does not mind this complexity, but the customer's invoice certainly does, and the model's attention quality often declines before the token limit is reached. Simply taking the last twenty messages as a controller action is not a viable long-term solution.

Microsoft.Extensions.AI 10.10 (experimental MEAI001) provides tools to address these issues: IChatReducer, ReducingChatClient, and UseChatReducer. This guide explains what breaks first and how to fix it.

A reducer works like this: ReduceAsync takes an IEnumerable of ChatMessage and returns a reduced IEnumerable. The inner client receives the reduced list for both normal and streaming calls. Reducing the list keeps the model focused on what matters most.

There are four common pitfalls to watch out for. First, tool pairs can become orphaned if not handled correctly. A FunctionCallContent without its matching FunctionResultContent (or vice versa) leads to provider 400 errors. Built-in reducers exclude tool call/result messages from the reduced view, but your raw store should keep them for debugging purposes. When writing a custom reducer, test with a mixed history to ensure neither content type survives unless you explicitly opt out.

Second, counting messages when finance tracks tokens can be tricky. MessageCountingChatReducer(targetCount) keeps the first system message and up to targetCount recent non-system turns. This approach is cheap and predictable for FAQ and short troubleshooting. However, lengthy stack traces can quickly exceed the billing limit. A custom TokenAwareChatReducer estimates token usage, always keeps the first system prompt, processes messages from newest to oldest until the budget is filled, and drops tool pairs by default. The implementation for this reducer is included in the complete project.

Third, summarizing with the flagship model can create confusion. SummarizingChatReducer(summarizer, targetCount, threshold) kicks in when unsummarized non-system messages exceed targetCount + threshold. Older turns are compacted into one assistant summary, while recent turns remain unchanged. The system prompt and tool content remain untouched.

Using the flagship model for summarization can inadvertently create an additional invoice while you focus on the initial savings. Also, summarization may insert a __summary__ tag in the AdditionalProperties of boundary messages. When previewing endpoints, clone messages first to avoid unintentionally modifying the live session store.

Finally, never forget the sacred system prompt. Every reducer variant mentioned above keeps it intact. If you decide to save tokens by silencing the system prompt, you essentially have a different chat product altogether. If the system prompt alone is large, consider optimizing the prompt rather than quietly dropping it in production.

To manage all of this, use a consistent session surface across API, worker, and CLI methods. A single pipeline suffices for creating sessions, handling chat messages, and retrieving raw or reduced transcripts. Simply swap the inner client for an OpenAI, Azure, or Ollama client as needed – the reducer layer remains unchanged.

Pin Microsoft.Extensions.AI to version 10.10 and treat MEAI001 as a usable-but-evolving package. The complete working project with counting, summarizing, and token-aware reducer modes, an offline summarizer, and session APIs is available in Tech Skill Builder for $5/month (normally $10). Don't wait – grab it now before the price increases.