{
  "id": 2606707,
  "title": "Speed Beat Relevance: What Broke When I Put an LLM in Front of Product Search",
  "url": "https://urgent.news/2026/08/22/speed-beat-relevance-what-broke-when-i-put-an-llm-in-front-of-product",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-22T15:35:32.000Z",
  "source": {
    "name": "HackerNoon",
    "slug": "hackernoon",
    "url": "https://hackernoon.com/speed-beat-relevance-what-broke-when-i-put-an-llm-in-front-of-product-search?source=rss"
  },
  "original_language": "en",
  "account": "In a recent experiment, a natural-language product search engine was built to handle listings in twelve different languages. The unexpected challenges stemmed from the interface between the model and the underlying systems rather than the language understanding itself.\n\nFirstly, a price filter that had never filtered correctly was discovered. The API's min_sale_price parameter was consistently ignored, filtering out only a few cents instead of the entire price range. This issue arose because the filter had never been tested with the actual minimum price value. A simple test would have caught the issue early on.\n\nSecondly, the model started generating valid-looking category IDs, even though some of them were incorrect. This was a form of hallucination that went unnoticed. To resolve this, the system was updated to validate every generated ID against the authoritative category tree before it was used.\n\nThirdly, a substring match in the query processing was found to be causing queries to be altered unintentionally. This resulted in queries like \"newborn baby clothes\" being matched to \"born baby clothes\" instead of the intended result. The fix was to ensure whole-word boundaries were respected in the matching process.\n\nLastly, the investigation revealed that latency costs were more significant than relevance. The search results took six to eight seconds to appear due to cache misses, which led to a high rate of drop-offs on the results page. By caching every search for 24 hours, the warm query time was reduced from seven seconds to a mere 0.2 seconds, leading to a significant improvement in conversion rates.\n\nThese findings emphasize the importance of testing assumptions, validating outputs, and considering the real sequence of events in a system when implementing new technologies like LLMs.",
  "summary": "Six failures from building an LLM-backed product search engine: a price filter that never filtered, invented category IDs, and latency that beat relevance.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}