Context engineering: the job prompt engineering grew into

Context engineering is the practice of carefully selecting and preparing the information a language model uses to generate responses to user queries. It involves deciding what data the model should see, such as the instructions, retrieved knowledge, system memory, tool results, and conversation history. Prompt engineering, on the other hand, focuses on crafting the instructions that guide the model's behavior.

In production environments, context engineering is more critical than prompt engineering because most of the context comes from automated sources like code and data retrieval systems. The term "context engineering" emerged as teams transitioned from single prompts to agents and retrieval systems, where much of the context is generated rather than manually written.

The main difference between context engineering and prompt engineering lies in the unit of work and the source of the information. In prompt engineering, the unit of work is the instructions written by a person, while in context engineering, the unit of work consists of everything the model sees for a request, including data retrieved from various sources.

A model's context comprises several components: the system prompt, request, retrieved knowledge, memory, tool definitions, tool results, task state, and other relevant information. Each of these sources has its own ownership, freshness, and permission rules, which are managed during the context engineering process.

As the amount of context increases, the performance of the language model may degrade. This is because the model struggles to focus on relevant information when presented with a large, unstructured context window. Long contexts can dilute the most important passages, and models tend to perform worse when trying to use information buried deep within a lengthy input.

To assemble context effectively for each request, it is essential to treat it as a structured build process with a budget. This approach involves setting token limits for different sections of the context, such as instructions, task state, memory, knowledge, and history. By allocating tokens wisely and filtering content based on user permissions, context engineering ensures that the model receives only the most relevant and permissible information.

When designing agents that utilize language models, it is crucial to manage the context carefully to prevent performance degradation. Some best practices include keeping task state separate from the model, summarizing tool results as they arrive, and rebuilding the context from state, memory, and fresh retrieval at each step. This technique, known as compaction, helps maintain the model's focus and prevents the accumulation of irrelevant information.