{
  "id": 17525723,
  "title": "Tool Selection at Scale: What Top-K RAG Leaves Out",
  "url": "https://urgent.news/2026/11/01/tool-selection-at-scale-what-top-k-rag-leaves-out",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-01T03:30:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/anuj-sadani/tool-selection-at-scale-what-top-k-rag-leaves-out-4aj9"
  },
  "original_language": "en",
  "account": "The article discusses the limitations of the \"embed, retrieve, decide and validate\" pipeline when dealing with a large number of tools, specifically at the 1,000 tool scale. The author argues that retrieval becomes an essential part of the process rather than just an optimization step. They compare this to the baseline pipeline's limitation to a single top-10 embedding lookup, which rarely misses at 100 tools but becomes necessary at 1,000 tools. The key points include:\n\n1. Tools are not just categories but have side effects such as sending emails, deleting records, or charging cards. Tool selection involves more than simple category classification.\n2. Retrieval helps control the context and improve accuracy, rather than just serving as a knowledge source. Position effects in long contexts can negatively impact accuracy.\n3. The recall chain is not a single number but the product of conditional recalls at each stage: retrieval, filtering, reranking, and selection. Improving retrieval can significantly boost overall performance.\n4. The focus should be on controlling the decision space, exposing the smallest context that still contains the right tool, rather than just maximizing retrieval recall.\n5. Measuring performance at each stage of the pipeline is crucial to identify which stage is the weakest link and requires tuning.",
  "summary": "Embed, retrieve, let the LLM decide works for about 100 tools. At 1,000 it needs a control plane: hybrid retrieval, hard filters, a learned reranker, progressive disclosure and per-stage evals. This is an adapted edition of the original article. It preserves the section argument and evidence; use the source link for the published version and any interactive material. Read the original article 01…",
  "key_points": [
    "Tools have side effects beyond simple category classification",
    "Retrieval controls context and improves accuracy",
    "Recall chain is product of conditional recalls at each stage"
  ],
  "editors_take": "Scaling tool usage to thousands of functions transforms retrieval from a simple optimization into a critical gatekeeper for managing context and preventing costly errors in automated decision-making pipelines.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}