Generation Got Cheap. Verification Didn't: What AI Actually Changed About Software Development

There are two groups of people when it comes to AI in software development. One group believes AI has made them ten times more productive, while the other group finds it unreliable and uses it sparingly. In my experience, both groups are correct, but they are measuring different aspects of the job. The main idea in this new workflow is that code generation has become cheap, but verifying it is still expensive.

Most of the benefits come from the fast code generation part, while the problems arise from the verification process. Starting a new project or task is now almost free, which encourages people to try more things and experiment with various designs. However, the real game-changer is reading and understanding existing code. AI tools excel at quickly providing information about a module's purpose and its calls, which saves a lot of time compared to manually searching through the codebase.

Nevertheless, the same tools are not very good at writing tests, as they can generate a large number of tests that simply assert the implementation does exactly what it currently does. Test writing still requires human judgment to determine what is truly important to assert. Code review has become a bottleneck, as pull requests are now longer and reviewers may become complacent due to the increased volume of code.

The bigger issue, though, is that AI-generated code often appears polished and well-written, making it harder for reviewers to spot potential mistakes. Reviewers are trained to spot human errors, and AI-generated code doesn't exhibit those telltale signs. As a result, junior engineers may have a harder time identifying issues in the code they produce.

Junior engineers often get the most value from writing small, scoped changes, test coverage, and understanding how a system works. These tasks, which were once valuable for building engineers, can now be automated by AI, leading to a loss of learning opportunities for junior staff. The author suggests that the most important rule now is to only merge code if they can explain it to a reviewer themselves.

This means keeping pull requests smaller and focusing on writing assertions for tests instead of letting the model do it all. The author believes that AI has shifted the value of this job from producing code to evaluating it, which is a more challenging and interesting task. Those who struggle with this change will be the ones who refuse to use AI tools, while those who succeed will be the ones who use them to improve their problem-solving skills.