{
  "id": 16859075,
  "title": "LoRA on a 16 GB card: what fits and what does not",
  "url": "https://urgent.news/2026/10/28/lora-on-a-16-gb-card-what-fits-and-what-does-not",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-28T06:33:20.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/izdravkov/lora-on-a-16-gb-card-what-fits-and-what-does-not-1ccc"
  },
  "original_language": "en",
  "account": "The maximum model size that can be fine-tuned using LoRA on a 16 GB consumer graphics card is approximately 7 billion parameters. This is not due to the model size causing errors, but rather because larger models consume more memory than the card can handle silently, leading to significant performance degradation.\n\nIn specific testing on an RTX 5070 Ti card with a 16,303 MiB VRAM limit, a 7 billion parameter model (rank 16, sequence length 512, batch size 1) reached a peak VRAM usage of 11,897.9 MiB, staying well within the card's limit. In contrast, a 14 billion parameter model completed training without any exceptions, but its peak VRAM usage rose to 22,189.8 MiB and 17,470.9 MiB, both exceeding the card's physical limit. This resulted in a dramatic slowdown of around 50 times compared to the 7B model, with no error messages indicating a failure.\n\nThe widespread belief that \"no error\" means the model fits within the memory constraints is incorrect. Windows automatically allocates system memory when a CUDA allocation exceeds the GPU's limit, causing hidden penalties and slower performance. For this specific setup, the true memory ceiling for Qwen2.5-Instruct, 4-bit base, LoRA rank 16 with a sequence length of 512 tokens and batch size 1, is 7B. Increasing any variable beyond this point pushes the model out of the card's memory capacity, regardless of whether gradient checkpointing is enabled or not. Therefore, on this particular hardware and configuration, a 14B model does not fit into a 16 GB card, even with gradient checkpointing.",
  "summary": "A 7B model is the honest ceiling for LoRA fine-tuning on a 16 GB consumer card, not because anything bigger throws an error, but because a 14B model silently spills into system memory and runs 50 times slower without ever reporting a failure. Key takeaways 7B is the real ceiling on an RTX 5070 Ti at LoRA rank 16, sequence length 512, batch size 1: 11,897.9 MiB peak, well inside the card's 16,303…",
  "key_points": [
    "7B parameters is max model size for LoRA on 16GB card",
    "14B model exceeds VRAM limit, causing 50x slowdown",
    "Windows allocates system memory, causing hidden performance penalties"
  ],
  "editors_take": "The tendency for Windows to automatically swap GPU tasks to system memory creates a deceptive training environment where performance collapses without providing any explicit error messages to the user.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}