1+1 is always 2: Bringing determinism back to AI-Assisted engineering

AI agents can now create functional applications from simple prompts, and automated coding demos can generate entire systems from scratch. However, the quality of the generated code is rarely demonstrated, as proof-of-concept stages do not focus on it. In production codebases, where many teams frequently make changes, code quality becomes crucial in determining system stability, maintenance costs, and whether engineers understand the code.

Among developers, 46% distrust AI tools' accuracy, while 33% trust them. Sixty-six percent cite output that is near-correct but not quite right as their primary frustration. An analysis of 211 million lines of changed code revealed that the percentage of refactoring changes dropped from 25% in 2021 to under 10% in 2024, while copy-pasted lines increased from 8.3% to 12.3% [1, 2].

The team behind this story builds AI workflows for a large monorepo with multiple projects, technologies, and domains. They found that AI agents become dependable in large codebases only when their structure around them is deterministic. This requires controlling the context the agent receives, the order in which work runs, and how its consistency is measured [3].

To achieve determinism, the team introduced path-scoped rules. Claude Code, one of the AI coding tools they use, automatically loads rule files when the agent reaches files matching specific path patterns [4]. As the agent moves through the codebase, relevant rules enter its context automatically. Each rule contains knowledge that the model cannot derive from the code itself, such as outdated patterns and outdated utilities in legacy code. The rules are kept short and simple, with each domain receiving its dedicated rule file [3].

To ensure consistency, the team employed LangGraph, an open-source framework for multi-step AI workflows. LangGraph provides determinism by executing its steps in a defined order every time and saving each step's state for debugging and rerunning [5]. The workflow begins with user requirements and an analysis of similar functionality in the code.

Then, the user confirms which projects the change affects, resulting in project-specific analyses and designs. These designs are combined into a single document for user review before being split into small and medium tasks for parallel implementation and validation. Every step runs in its own agent session with minimal context to prevent details from being lost [5].