Telling a cheap model to "think a little" made it slower and sometimes silent
I told a model to think less, and it started thinking. That's the whole bug. But how I got there is worth writing down, because anyone putting one endpoint in front of several model families will probably hit some version of it. The setup: I build a desktop app that can talk to a model three ways. A local model through Ollama, the user's own API key, or cloud credits that go through a small…
A software developer discovered that limiting the reasoning capabilities of certain models can lead to slower performance and silent responses. The issue arose when building a desktop app to communicate with multiple model families through different endpoints. The gateway that handles these connections was designed to cap reasoning for models that think by default, such as Gemini 2.5.
However, the developer found that models like Gemini 2.5 Flash-Lite, which are designed to be cheap and quick, would still engage in reasoning even with a low effort setting. This resulted in most of the token budget being spent on thinking, leaving no tokens for a meaningful response. The original code used a regular expression to identify reasoning-capable models, but it failed to account for model variants that don't think by default, such as Gemini 2.5 Flash-Lite.
To fix the issue, the developer introduced a negative lookahead to specifically exclude the Gemini 2.5 Flash-Lite model from receiving reasoning caps. Additionally, they added a test to verify that non-reasoning models are not incorrectly assigned reasoning capabilities. The core lesson from this experience is that model families cannot be relied upon to consistently behave in the same way, and relying solely on model names or families is unreliable.
Instead, the developer recommends verifying exact model IDs and testing with small max_tokens values before routing requests to latency-sensitive endpoints. This story was reported based on the original source material, with no additional information or speculation added.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
