Urgent.News

One page, thousands of outlets. See who else covered it.

Editions

AI

Invoked, not executed

A deep-research request to the top-tier model tore through three consecutive five-hour usage windows, the rolling quota Claude enforces before a session has to stop and reset, to answer a single question. Usage window Burned Time to burn it Window 1 54% about two minutes Window 2 100% about four minutes Window 3 40% finishing the job Three consecutive five-hour usage windows against one question.…

A deep-research request to the top-tier model consumed three consecutive five-hour usage windows to answer a single question. Two of those windows were exhausted in under six minutes, burning nearly two entire windows' worth of time. The question, asked directly, required the model to search, fetch sources, extract claims, and verify them objectively.

The model's routing policy directs such tasks to the cheapest tier that can accomplish them, not the most expensive one. The routing policy activates only at decision points during a session, but direct requests, like "deep-research this," skip that step, causing all processing to occur within a single reply from the most expensive model.

To fix this issue, the model should route research-shaped requests through a harness that breaks them down during input. This would apply to all three types of requests: explicit instructions, model decision-making, and soft inquiries. Six subagents were tested with three different phrasings of the same research question, using two model tiers.

The harness successfully triggered in half of the runs, but only because some subagents improvised their own fan-out step. The harness lacked the necessary functionality to handle the research properly, as it could not be called within a delegated subagent. The explicit "deep-research this" phrasing did not work due to the routing logic, and one successful run explained why it performed as it did.

Using the phrase "deep-research this" reliably invoked the harness, but rephrasing the request without those specific words stopped the coincidence that made the easy case appear solved. The most critical case involves a serious, specific question asked without research-related vocabulary, which is the typical shape of real research needs.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in AI

More from Thursday 20 August →