Context engineering is mostly deciding what to leave out
Most advice about context windows is about fitting more in. Bigger windows, more retrieved documents, the whole file instead of the function. In practice the thing that has improved my results the most is the opposite move: getting the junk out. A window packed with marginally relevant material makes the model answer worse, not better, and you pay for every token of it on every turn. More context…
Context engineering is largely about removing irrelevant information. Advice often revolves around fitting more data into a window, but the most impactful change is often eliminating unnecessary material. A window filled with somewhat relevant content actually hinders a model's performance. The model processes all provided context, not just the relevant parts, so irrelevant data competes for attention.
Adding five pieces of retrieved data in a context instead of one does not add four new data points, but four potential distractions from the one piece that matters. Long contexts also have a tendency for the middle content to be weighted less than the beginning and end, affecting whether facts get used. A smaller, more focused context often yields better results than a larger, more comprehensive one, even if the larger one technically contains the correct answer.
When deciding what to include in the window, think of three categories: the task itself, the few facts directly related to the task, and the desired output format. Almost everything else should be considered for removal. The most common things I remove are: the entire file when only a few lines are relevant, the entire conversation history when only the last couple of turns are relevant, boilerplate headers and licenses from retrieved snippets, and documents scored as somewhat relevant but a human would deem off-topic.
Retrieval gives you candidates, but the crucial part is narrowing down to the ones that truly matter. Placement matters too, as the start and end of a long context receive the most attention. The instruction and most relevant facts should be near the end, close to the question. When constructing prompts from multiple sources, prioritize the sources based on their relevance to the task, not the order of retrieval.
The cost is the same - every token added is a token you pay for, and in a multi-turn agent, you pay for the growing history again on every subsequent call. Therefore, unnecessary context is costly in both money and quality. This makes trimming a valuable strategy that can both improve answers and lower costs simultaneously. The practical approach involves a short loop: retrieve broadly, then re-rank and trim to the few pieces that truly matter.
Summarize lengthy history instead of resending it raw, and strip out boilerplate before it enters the prompt. If an answer is incorrect, examine the window before increasing the model size, as often the solution is not more context, but less of the incorrect context. Sharing examples of successful prompt adjustments can provide valuable insights for others in building with these models.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
