Model Routing Is Simple. Until It Isn’t.
At first glance, creating a routing system for your agent might seem straightforward. You send simple requests to cheaper models, reserve more powerful ones for complex tasks, and use a classifier or heuristic to make the decision. This leads to cost savings and maintains performance. However, this is not the full story. Most routing systems assume that model selection is a simple classification problem.