AI Coding Agents Changed What I Review in ML Apps

In recent months, my approach to creating machine learning applications using coding agents underwent significant changes. Initially, the process entailed writing a prompt, allowing the agent to modify the code, opening the diff, examining the changes, running the code, and repeating the cycle. The initial success was due to the simplicity of the application and my ability to maintain an understanding of the entire codebase.

However, as the application evolved, the agent's modifications became more extensive, affecting several modules and introducing hundreds of lines of code. The constant review of lengthy code changes became cumbersome and ultimately became a bottleneck in my workflow.

I realized that I had unintentionally become the bottleneck, spending a considerable amount of time reviewing the agent's implementation rather than focusing on the application's output. It became apparent that I was tackling the wrong problem. I was concerned about controlling the means (the code) instead of the end goal (the working application).

The black box nature of the code review process became evident. Once I ceased reviewing the code, there was little insight into the application's behavior, only a vague representation of "Input → [ ? ? ? ? ? ] → Output."

To address this issue, I began asking the agent to structure the application into meaningful, explicit steps. For example, I transformed the code into a pipeline:

Pipeline([ Preprocess(...), Inference(...), Postprocess(...), ApplicationLogic(...), ])

While the agent continued to generate the implementation behind these steps, I no longer relied on the entire codebase to comprehend the computation. The critical aspect shifted from merely reviewing the final output to inspecting intermediate results, providing a clearer understanding of how the data flowed through the computation. This shift proved particularly valuable for machine learning applications, as the intermediate data often offered more insight into correctness than the implementation.

The changed approach also influenced how I utilized the coding agent. Now, I asked the agent to structure the code in a way that allowed for easy inspection. The interaction between the agent and me became more harmonious, with the agent handling implementation details within the defined steps. This change allowed me to focus on the application's performance, concurrency, resource management, and security aspects at a higher level while still having the ability to debug specific problematic steps.

Though code review remains essential, I no longer prioritize it as the first action. Instead, I first assess the data flowing through each step and the resulting output. By identifying where the behavior deviates from expectations, I can effectively communicate the issue to the agent, rather than requesting a complete application debugging session.

Ultimately, this transition from code-centric to result-centric development emphasizes making ML applications more inspectable, where the crucial transformations are explicit, and the intermediate results can be effectively reviewed.