The LLM Suggests; Code Decides: Building a Cost-Aware Router for Image and Video Models

AI model routing is a seemingly straightforward process: read the user's prompt, consult an LLM to determine the most suitable generation model, and execute the generation. However, this approach quickly becomes unreliable when the product supports various tasks like text-to-image, image editing, multi-reference fusion, and video, each with distinct aspect ratios, durations, pricing rules, and access tiers.

The key learning is to view the LLM as a planner, not as the final authority, and to implement a constrained pipeline for decision-making.

The first step is to route tasks rather than model names. Before selecting a model, the system must identify the actual generation task. The types are: image-to-image, image-to-video, image fusion, and video. Each task has unique requirements and constraints that significantly influence the model selection process.

Next, utilize rules to establish strong signals for deterministic decisions and employ an LLM only when ambiguity arises. The order of classification matters. For example, "Create a cinematic 3x3 photo grid" should remain classified as an image task, while "Animate this photo into a short cinematic video" is explicitly an image-to-video task.

Strong signals should be deterministic, while genuinely ambiguous requests should rely on the LLM for classification. Also, record whether a decision came from rules, LLM, or fallback for easier debugging.

Lastly, model capabilities should be treated as data, not prompt text. The planner should not depend on its training data to remember model constraints. Maintain a catalog that code can validate, containing details like model ID, display name, supported generation kinds, minimum tier, base credits, supported resolutions, aspect ratios, supported formats, and tags. The LLM should receive only the models available for the current task and membership tier, allowing it to make informed decisions based on the available options.