{
  "id": 6612649,
  "title": "HNSW ef_search: Why Your Vector Search Misses the Right Chunk",
  "url": "https://urgent.news/2026/09/10/hnsw-ef-search-why-your-vector-search-misses-the-right-chunk",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-10T16:43:32.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ji_ai/hnsw-efsearch-why-your-vector-search-misses-the-right-chunk-19a4"
  },
  "original_language": "en",
  "account": "The vector index does not guarantee returning an existing vector. HNSW (Huang, Nello, Scott & Weiss) is an approximate nearest-neighbor index that walks a graph built over vectors, giving up early on purpose. ef_search determines the size of the candidate list for this walk. If your LIMIT is near ef_search, recall drops dramatically. Metadata filters can worsen this issue. To measure recall@k, run the same queries with an exact scan and compare ID sets. HNSW does not search vectors; it walks a graph with a fixed candidate list size. As corpus grows, embeddings cluster, and ef_search shrinks, the failure becomes more likely. The walk is greedy, moving to the closest neighbor and dropping layers when it can't improve. The failure is silent and query-dependent. To improve recall, increase ef_search, build time parameters m and ef_construction, and periodically reindex. Measure recall@k by comparing ID sets from exact and approximate scans. Plot recall@k and latency to find the optimal ef_search value.",
  "summary": "The chunk was in the index. I checked three times. A support bot kept answering \"that isn't covered in the documentation\" for a question that was covered, in one paragraph, in a doc we had ingested two weeks earlier. I pulled the row out of Postgres by ID. It was there. Embedding present, 1536 dimensions, right tenant, not soft-deleted. I ran BM25 over the same corpus and the passage came back at…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}