Query decomposition doesn’t fix context starvation — it just moves it
I have a small example that would best communicate the message I am trying to convey: say you built a The post Query decomposition doesn’t fix context starvation — it just moves it appeared first on The New Stack .
The article explores the issue of context starvation in Retrieval-Augmented Generation (RAG) pipelines, which occurs when a specific piece of information from the retrieved passages is not allocated enough space in the final packed context, even if that evidence was successfully retrieved. The author uses the example of a chat widget for GitLab's public documentation, where a developer asks five separate questions related to payment decline, seat overage, compute-minute rollover, changing invoice recipients, and API rate limits.
By decomposing the message into five single-intent queries and running them through a RAG pipeline, the pipeline retrieves all five correct passages, but only one of them survives into the packed context, while the others are dropped. This demonstrates the context starvation phenomenon, where a sub-intent that gets no allocation in the final packed context, even if its evidence was successfully retrieved.
The author calls this context starvation because it is the absence of allocation, not the absence of retrieval, that is the problem. The failure moves from the embedding stage, where it has a name and people watch for it, to the packer stage, where it has neither. The author suggests that the current practice of splitting the query into sub-queries and packing the highest-scoring passages into a fixed token budget is a priority scheduling algorithm with no aging term, which leads to starvation of low-priority sub-intents.
Written by urgent.news from The New Stack's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
