Your AI coding agent could cost 5x more depending on the harness, benchmark finds
The same AI model can cost up to five times more to run depending on the coding agent harness it’s The post Your AI coding agent could cost 5x more depending on the harness, benchmark finds appeared first on The New Stack .
A new benchmark conducted by The New Stack has found that the cost of running an AI model for coding agents can vary significantly depending on the harness used. The benchmark tested 22 configurations across five open-weight models, consuming approximately 83 billion tokens. The model itself accounted for 92.8% of the cost differences, but the harness still had a considerable impact.
Nearly 65% of the estimated inference spending went to runs that failed verification or timed out. The rankings of different harness configurations shifted significantly when the AI model changed, with a median of 11 ranking positions changing across models. Failed runs accounted for 52.7% of estimated inference spending, with timeouts adding another 11.8%.
Brief written by urgent.news from The New Stack's own syndicated text. Machine-written — may contain errors; check the original before relying on it.