Writing the code is no longer the bottleneck

The coding industry is currently fixated on the speed of generating code. Daily, engineers post links to new agents or models capable of producing thousands of lines of flawless frontend or backend code within seconds. Teams treat implementation speed as the main obstacle to software delivery, but it's not. Implementation is mostly solved.

Generating a new microservice for regional tax calculations in our checkout flow can be completed with generated scaffolding, logic, and unit tests before coffee cools. The real bottleneck has shifted to ensuring the generated code is correct. Evaluation has become the most challenging aspect of our jobs. I witnessed this when a team optimized indexing product metadata into the search database using an automated workflow.

The engineer refactored a complex part of the ingestion pipeline in an afternoon using AI. The code was clean, followed our patterns, and had passing unit tests. However, after deployment to staging, the search results for the UK region behaved unexpectedly. It took six days of manual log analysis to discover that the generated code misunderstood how to handle VAT-inclusive pricing in the search index.

While the code was technically correct, it lacked the years of experience embedded in our legacy order management system. This highlights the need to treat evaluation as a first-class engineering discipline when using AI at scale. Currently, most teams still rely on unit tests and superficial reviews. Metrics like tickets moved to "In Review" and lines of code shipped are outdated.

In this era of free code, these metrics are dangerous as they encourage flooding repositories with unchecked code. The focus now should be building robust testing frameworks, including shadow deploys, complex regression suites, and observability mechanisms that can detect logical errors before they reach production. I am not immune to this.

Last Tuesday, I approved a new internal tool built almost entirely with AI agents for a pilot without thoroughly evaluating its testing plan. I let the allure of the fast implementation speed overtake the need for a proper evaluation framework. I fell prey to the firehose of unverified code, adding more cognitive debt to a team already overwhelmed with existing services.