The Intelligence Ladder

The article discusses the need for an intelligence ladder, a system that sorts work into rungs and specifies which model and roughly how much it costs for each rung before work begins. The author, who has been grappling with the question of how much intelligence is needed for a particular task, argues that the current fuzzy answers given by AI models lead to a sunk cost trap, where users continue spending money without a clear understanding of the true cost.

The lack of accountability in the industry is likened to a builder quoting a kitchen or a printer quoting a run of five thousand flyers, but the coding agent industry normalizing fuzzy estimates. The author also raises concerns about access to AI tools, questioning whether only the wealthy can afford to build software using these tools.

The potential for a world where only the wealthy can build software is highlighted as dangerous. The author proposes a solution in the form of a behavior-based system, where work is broken down into shareable pieces called behaviors, which can be composed to create a program. Each behavior is assigned to a rung on the intelligence ladder, and the model choice and cost fall out of this placement instead of being guessed.

The author demonstrates this with a real-time chat app example, which consists of five behaviors composed from shared components used throughout the library. The goal of this system is to eliminate guesswork and provide a more transparent and accountable process for using AI models in software development.