{
  "id": 3427543,
  "title": "Upgrading Your Embedding Model Doesn't Break RAG Loudly — It Breaks It Quietly",
  "url": "https://urgent.news/2026/08/26/upgrading-your-embedding-model-doesnt-break-rag-loudly-it-breaks-it",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-26T04:38:34.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/speed_engineer/upgrading-your-embedding-model-doesnt-break-rag-loudly-it-breaks-it-quietly-ih6"
  },
  "original_language": "en",
  "account": "A team I was assisting upgraded their embedding model to reduce costs by switching from an older general-purpose model to a newer, cheaper one. No schema changes, downtime, or errors were observed in the logs. However, over the subsequent three weeks, support tickets increased as the assistant began confidently providing incorrect documents. The team did not connect the issue to the embedding model change, as the system continued functioning without crashing. Retrieval systems do not throw errors when they provide incorrect information; they simply return the most similar vectors, which have stopped having any meaning.\n\nThe issue arises because embedding spaces are not portable across models. Even if two embedding models output 1536-dimensional vectors and are both excellent, they may still be incompatible. This is because each model learns its own geometry during training, influenced by its own objective and data. There is no shared coordinate system or translation layer, so two models may not agree on what \"close\" means. When only new documents are re-embedded with the new model, while old vectors remain in the same index, you end up with a vector store containing vectors from both models. A query embedded with the newer model gets compared against vectors from both models. While cosine similarity is meaningful when compared to model-B vectors, it becomes closer to noise when compared to model-A vectors, with no reliable relationship to actual semantic relevance.\n\nTo demonstrate the severity of this issue, a quick test was conducted. The cosine similarity of vectors from the same model for related concepts clustered tightly and had a high similarity (0.83). However, when comparing a query embedded with the newer model to vectors embedded with the older model (from weeks ago), the similarity scores varied (0.71, 0.44, 0.79, 0.52, 0.68), showing no relationship to actual relevance. These cross-model similarity scores were not uniformly bad, which is the trap. Some high scores occurred by coincidence, which is worse than all of them being low because a high score that means nothing still gets retrieved with confidence and handed to the LLM as relevant context. The model does not hesitate on garbage context and generates fluent, confident answers based on a document that was never actually related to the question.\n\nTo address this issue, it is essential to treat embedding model changes like schema migrations, not config tweaks. Here are a few recommended practices that work in production:\n\n1. Perform a full reindex instead of incremental backfill. If the model changes, every vector in the index needs to be re-embedded with the new model. Partial migrations lead to the exact failure mode described above, where a two-model index looks fine but is not.\n2. Version-tag every vector's metadata with the embedding model name and version. This costs only one field and allows you to query how much of your index is stale instead of guessing.\n3. Shadow-evaluate the new index before flipping to it in production. Stand up the new index in parallel, run a fixed evaluation set of real queries through both models, and compare retrieval@k and answer quality. This step is often skipped due to time pressure but is crucial to avoid silently corrupting retrieval results.\n4. Never mix embedding models in one index, even temporarily. Temporary mixing is when nobody is watching, leading to silent corruption of retrieval results.\n5. Batch the re-embedding cost down and do not skip it. Queue it, rate-limit it, and run it overnight. The cost is still cheaper than a support queue full of confidently wrong answers.\n\nIn summary, different embedding models produce vector spaces that are not comparable, even if they have the same dimensions. Mixing vectors from two models in one index does not fail loudly but silently corrupts a subset of retrieval, sometimes convincingly. Treating embedding model upgrades as full-index migrations with version tags and shadow evaluations, rather than drop-in model swaps, can prevent this issue.",
  "summary": "The problem A team I was helping upgraded their embedding model to cut cost — swapped an older general-purpose embedding model for a newer, cheaper one. No schema change, no downtime, no errors in any log. Over the next three weeks, support tickets crept up: \"the assistant is confidently answering with the wrong doc.\" Nobody connected it to the embedding swap because nothing had crashed.…",
  "key_points": [
    "Upgrading embedding model to cheaper one reduces costs without schema changes or errors.",
    "Retrieval systems return most similar vectors even if incorrect, no errors thrown.",
    "Mixing embedding models in one index silently corrupts retrieval results over time."
  ],
  "editors_take": "Upgrading an embedding model can silently corrupt retrieval results if not managed as a full-index migration, leading to confidently incorrect answers being provided without noticeable system errors or downtime.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}