Engineering Reliability into AI Agent Code Generation. Part II

Part II of this report delves into the intricate components of the AI agent code generation architecture, providing a detailed examination of each element. The architecture is mapped onto the agentic-patterns catalog, and a critical boundary is identified: models generate and evaluate, while deterministic code decides.

The report presents three proposed patterns, written in catalog form, and explores the third component - deterministic guards. It highlights the challenges of an LLM orchestrator exhibiting P2, P3, and P7, and presents a countermeasure in the form of a state machine that prohibits the orchestrator from making decisions regarding phase transitions.

Additionally, the report introduces the evidence model, an artifact taxonomy that demands proof for every claim made by agents. It explains the adversarial nature of evidence design, as agents have the ability to edit tests to pass them, and checks require their own test suites. Context engineering is also discussed, focusing on briefs assembled by scripts under strict budgets and an orchestrator that sees only paths and states.

The report further explores adversarial evaluation, presenting a ladder of fail-fast mechanical checks and itemized design checklists as alternatives to vague vibes. It introduces bounded retries with structured escalation and a connector contract that closes the "false-green" class, normalizing at the boundary, earning capability flags, and ensuring cross-witness completeness. The concept of SKIPPED is never considered a PASS.

Traceability is presented as a computed property, and buying rigor is discussed by the unit, with routing by stakes and an honest cheap path being proposed. Escalation is emphasized as a first-class state. The report concludes by outlining traceability as a computed property, unit-based rigor, bounded retries, and the connector contract as the key components to ensure engineering reliability into AI agent code generation.