{
  "id": 7020817,
  "title": "I described 1,245 tables with an LLM and retrieval got worse",
  "url": "https://urgent.news/2026/09/12/i-described-1-245-tables-with-an-llm-and-retrieval-got-worse",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-12T23:35:28.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ashish_sinha_5241c7673d93/i-described-1245-tables-with-an-llm-and-retrieval-got-worse-58a"
  },
  "original_language": "en",
  "account": "An LLM was used to generate 1,245 table descriptions, which were then indexed alongside table names to improve retrieval. However, the recall dropped significantly, with the \"contacts\" table falling below rank 40 in search results. The issue lies in the correlated vocabulary between table descriptions and user queries. When the descriptions are generated, they use the same vocabulary as the user queries, causing a phenomenon called IDF collapse. This makes the most frequent terms, like \"contacts,\" less informative and lowers their scores. Additionally, length normalization penalizes long documents, making tables with many columns, like \"contacts,\" appear more relevant. This creates a compounding effect, where IDF collapse removes the signal that separates relevant tables from others, and length normalization further prioritizes the most important tables. Cataloguing, while useful, adds correlated noise that attacks the very queries it aims to help. The optimal solution is to score fields separately and combine the rankings. This approach involves creating three separate BM25 indexes: one for the entire document, one for the table names, and one for the generated descriptions. By scoring the fields independently and combining the results, the model can better differentiate between relevant tables and minimize the impact of correlated noise.",
  "summary": "The cataloguing step is supposed to be the easy win. You have a schema whose tables are called ecm_template_link and v_pmpm , your users ask questions in English, and the gap between those two vocabularies is why retrieval misses. So you point a model at every table, get back a sentence describing each one, index the sentences alongside the names, and now the corpus speaks English too. I did that…",
  "key_points": [
    "LLM generated 1,245 table descriptions for retrieval improvement",
    "Recall dropped significantly, contacts table fell below rank 40",
    "IDF collapse and length normalization complicate table retrieval"
  ],
  "editors_take": "Adding correlated table descriptions generated by a large language model to improve retrieval actually hurts search results by introducing noise that overshadows relevant information, requiring a more nuanced scoring approach.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}