Where the time actually goes in an AI coding-agent job

In the world of AI-powered coding agents, the time spent on a typical job is often dominated by infrastructure costs rather than the actual model inference. A standard job, which typically takes 20-30 minutes, begins with the creation of a git worktree, followed by a cold setup process that takes around 1-2 minutes. This is due to the need to install dependencies, run linting and typechecking, and build the application, all of which are performed from a cold state every time.

The Vitest runner, used for testing, takes approximately 26 seconds, which is fast and acceptable for the job. The model time, which encompasses planning, execution, and review calls, accounts for the majority of the remaining runtime - 15-22 minutes.

However, a significant portion of the total runtime can be attributed to infrastructure overhead, which accounts for 20-30% of the total job time. This includes the cost of setting up a new worktree each time, cold dependency installation, and the building of the .next directory. By eliminating this overhead, the time for small jobs can be reduced from 30 minutes to 20 minutes. The remaining time is then strictly bounded by the model inference, making it the phase worth waiting for.

To optimize this infrastructure pipeline, several strategies can be employed. Firstly, a pre-warmed executor slot pool can be maintained. This eliminates the need to perform a fresh installation for every job, thereby cutting down the initial setup time. Secondly, persistent Next.js build caching can be utilized. This transforms the longest gate step from minutes to just a few seconds for standard diffs.

Thirdly, incremental linting and typechecking can be enabled, which saves roughly 1 minute per job. Fourthly, Windows filesystem optimizations, such as excluding worktrees from real-time antivirus scanning and moving worktrees to a ReFS Dev Drive, can significantly improve I/O speeds. Lastly, isolated linkers can be used, although this requires caution due to the risk of unintended deletions.

By implementing these optimizations, the fixed job overhead can drop from 5-8 minutes to under 60 seconds. The remaining runtime, which represents actual model reasoning, becomes the phase worth waiting for.