{
  "id": 15706280,
  "title": "Catalog Matching With LLMs: Why Changing Knowledge Should Stay Outside the Model",
  "url": "https://urgent.news/2026/10/20/catalog-matching-with-llms-why-changing-knowledge-should-stay-outside",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-20T05:40:17.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/olegkokorin/catalog-matching-with-llms-why-changing-knowledge-should-stay-outside-the-model-4pkn"
  },
  "original_language": "en",
  "account": "Catalog matching with large language models (LLMs) can appear straightforward, but real-world data introduces complexities. A single product may have various names, abbreviations, languages, units, brand conventions, and category structures. Traditionally, an automated system would need a locally trained model on historical matches, deployed on dedicated infrastructure, and periodically fine-tuned as new errors emerge. While this approach is viable, it raises questions about how the system should incorporate new naming conventions, supplier-specific terminology, and matching rules over time. If each new pattern requires another training cycle, model adaptation turns into a constant ML maintenance process.\n\nThe core issue is whether changing domain knowledge should be integrated within the model weights or kept separate. One training cycle is insufficient because the same entity can be represented differently across catalogs. Inconsistencies may include abbreviations, transliteration, mixed languages, different brand naming conventions, measurement units, product classification systems, new attributes, missing attributes, and supplier-specific terminology. While some of these inconsistencies can be addressed during data normalization, new cases will still arise that weren't present in the initial training data. Incorporating every new variation into the model itself quickly transforms into a full ML lifecycle, necessitating case collection, labeling, preparation of new training data, fine-tuning, evaluation, and deployment of a new model version.\n\nAn alternative approach involves keeping the underlying model stable and altering the context it receives when making decisions. When the system encounters a new product pair and cannot confidently determine a match, the case is sent for human review. The reviewer confirms or rejects the match, and the validated example is stored in a database of previous decisions. Subsequently, when a similar case appears, the system retrieves relevant validated examples and presents them alongside the new product pair to the LLM. The model no longer evaluates the case in isolation but also considers how similar situations were handled earlier. This method, known as retrieval-based system adaptation rather than model retraining, entails no fine-tuning of the LLM weights. Instead, the external knowledge layer available at inference time evolves, allowing for more rapid adaptation to new data.\n\nThis strategy becomes particularly effective when a new pattern emerges. For instance, if a known brand suddenly appears under a spelling not present in the original dataset, a fine-tuning workflow might save that case as a new training example and include it in the next model update. However, a single new example rarely justifies an immediate fine-tuning run. More data typically needs to be collected, and the model retrained, evaluated, and deployed as a new version. With retrieval-based adaptation, the validated example can be utilized almost immediately. Once a human confirms the relationship, it becomes accessible to future requests, making it particularly useful when the new information represents knowledge or a business rule rather than a new capability for the model to learn. There's little need to modify model weights merely to teach the system that \"EVOO\" means \"Extra Virgin Olive Oil.\" Similarly, supplier-specific knowledge, such as a retailer using \"JD\" as shorthand for \"Jack Daniel's,\" can be stored externally and inserted into the context only when relevant. This architectural separation keeps the LLM focused on language understanding and reasoning based on provided information, while the retrieval layer maintains the current knowledge about a specific catalog and its matching rules outside the model. Correcting errors in this system is easier to control since the connection between newly added knowledge and specific decisions is usually clearer. Validated cases retrieved by the model can be inspected, edited, or removed if incorrect. Updating business rules becomes simpler without requiring a new model version, as the layer containing accumulated domain knowledge remains separate and more manageable.",
  "summary": "Product catalog matching looks simple until real-world data enters the picture. The same product may appear under different names, abbreviations, languages, units, brand conventions, or category structures. A human can usually recognize these variations, but an automated system has to formalize them. One obvious approach is to train a local model on historical matches, deploy it on dedicated…",
  "key_points": [
    "Changing domain knowledge should stay outside LLM model.",
    "Retrieval-based adaptation allows rapid incorporation of new data.",
    "Validated examples can be edited or removed if incorrect."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}