AI Can Write Code Fast, but Testing Is Still the Bottleneck

AI has revolutionized the way the writer approaches their mini-game development. With tools like GPT, Gemini, and Codex, they can implement features much faster than before. These AI-powered tools assist in writing code, generating configurations, reviewing logic, fixing compilation errors, designing game mechanics, and maintaining documentation.

Recently, the writer used AI to work on various game elements such as enemies, bosses, and their associated features like speed bursts, armor changes, summoning, healing, and different boss behaviors. This has led to a significant boost in productivity.

However, the writer has discovered that writing code is no longer the biggest bottleneck in their development process. Instead, testing has become the limiting factor. While AI makes implementation cheap, testing remains a time-consuming and crucial aspect of game development. Despite AI's ability to handle a significant portion of coding tasks, it cannot replace the need for actual testing and validation.

The writer highlights that just because a feature is easy to implement with AI, it doesn't mean it is finished. A game mechanic needs to be experienced within the game to ensure it functions as intended. Issues like enemy speed, boss healing, summoned enemies' behavior, ability triggers, and level pressure must be thoroughly tested.

The testing process involves launching the game, observing the mechanic, recording the results, and making adjustments as needed. This requires dedicated testing time, which can be challenging for the writer due to a busy schedule.

The writer refers to this imbalance between code production and code validation as "testing debt." AI can indeed create more testing debt, as implementing multiple features becomes tempting when coding is quick and easy. While each new feature may seem inexpensive individually, the overall project progress is hindered if only a fraction of them are properly tested. The writer emphasizes that maximizing AI output does not necessarily lead to project progress.

To address this issue, the writer is actively building tools to make testing faster and more efficient. They have added debug commands that allow for quick spawning of specific enemies, bosses, or game states, bypassing lengthy gameplay sessions. The goal is to streamline the testing process by jumping straight to the desired scenario for validation.

The ultimate workflow for each mechanic would involve launching the game, entering a debug command, immediately setting up the target scenario, observing the mechanic, recording the results, and adjusting as necessary. AI can assist in building better debugging tools, test commands, state displays, and shortcuts to optimize the testing process.

However, the writer also acknowledges that AI cannot solve the challenge of deciding when something is good enough. A game can always be fine-tuned, with enemies moving faster, bosses having less armor, levels containing more enemies, or abilities triggering earlier. For a side project, trying to perfectly tune every number before release is not realistic due to limited time.

The writer aims to distinguish between release blockers, such as broken mechanics, game-breaking bugs, serious balance issues, or incorrect behavior, and other aspects that can be iterated upon later.