{
  "id": 20544218,
  "title": "Your LLM app has an expiry date: pin the model and keep a migration eval ready",
  "url": "https://urgent.news/2026/11/18/your-llm-app-has-an-expiry-date-pin-the-model-and-keep-a-migration",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-18T04:27:48.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/m_t_ramkrushna/your-llm-app-has-an-expiry-date-pin-the-model-and-keep-a-migration-eval-ready-1p2f"
  },
  "original_language": "en",
  "account": "Calling a specific model version isn't a good strategy; it's a dependency you haven't pinned. Every LLM feature you deploy will rely on a model that might change or be switched off. Tackle the model like any other production dependency: pin it, monitor it, and prepare for migration before it's retired. The same model doesn't stay the same. A study comparing ChatGPT's behavior showed significant changes in accuracy and code generation capabilities between March and June 2023 versions of GPT-3.5 and GPT-4. Both models made more formatting mistakes in code generation in June than in March. This highlights the need for continuous monitoring of LLM behavior. Pinning is easy now. Use it. Providers give you stable identifiers for their models. Anthropic's models overview lists every Claude model ID as a pinned snapshot, including dateless IDs used since the 4.6 generation. OpenAI also provides pinning options, but with different deprecation policies. In production, always send an ID that names exactly one model. Aliases and -latest names are fine for testing but not for production use. However, pinning alone isn't enough. A pinned model doesn't live forever. Anthropic issues retirement dates, and requests to models past the retirement date will fail, giving customers at least 60 days notice. OpenAI provides at least 6 months notice for generally available models and 3 months for specialized variants. Real-life examples show that model deprecations can happen with short notice, turning a planned migration into a weekend incident. The key is your evaluation set. Both providers stress testing your applications with newer models well before the retirement date. Create a file of real inputs from production, redacted, with expected properties or rubrics. Run representative requests against both models, comparing outputs, latency, and cost before switching production traffic to the replacement. If you don't have an eval set, a forced model swap means manually checking a handful of prompts and hoping. The process should be like upgrading any other dependency: pin the model ID in config, not scattered across code, log the model ID with every request, keep a migration eval set, watch for shutdowns programmatically, and roll out like a dependency upgrade with canary testing. This approach ensures model upgrades are manageable and doesn't catch you off guard.",
  "summary": "Wednesday AI Opinion. Here's my hot take: \"we use the latest model\" is not a strategy. It's a dependency you haven't pinned. Every LLM feature you ship is built on a model that will either change under you or get switched off. Treat the model like any other production dependency: pin it, watch it, and keep a migration eval ready before the retirement email shows up. The \"same\" model doesn't stay…",
  "key_points": [
    "Pin the specific model version, not just a version number, to avoid dependency issues.",
    "Both Anthropic and OpenAI provide model IDs for pinning, with different deprecation policies.",
    "Create and test an evaluation set with real production inputs before migrating to a new model."
  ],
  "editors_take": "Treating large language models as volatile dependencies rather than static tools forces developers to adopt rigorous version pinning and continuous evaluation to prevent sudden production failures during inevitable model retirements.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}