Model routing with OpenRouter and DeepSeek to cut costs without losing quality

Model routing is a technique where requests are sent to the cheapest model that can still achieve the required quality, rather than sending everything to a single expensive model. In a production implementation using OpenRouter, DeepSeek is used for the majority of traffic (classification, extraction, summarization), while a premium model like Claude is reserved only for cases where quality is critical.

This approach cuts AI costs significantly, often by 80-90%, without users noticing a difference. The process involves sending the bulk of traffic to a cheaper model like DeepSeek, validating its output, and escalating to the premium model only when necessary. It's crucial to clean the JSON before giving up to avoid unnecessary escalations.

To determine which tasks the cheap model can handle effectively, you need to identify tasks with predictable, verifiable formats (like classification, extraction, summarization), where DeepSeek performs near-par with premium models. However, tasks requiring open-ended reasoning, strict adherence to long instructions, or nuanced writing may need the premium model.

This strategy reduces costs by directing only the most complex tasks to more expensive models, while the majority of requests are handled by cheaper models.