{
  "id": 3255178,
  "title": "Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original",
  "url": "https://urgent.news/2026/08/25/quantization-aware-healing-a-compressed-4-bit-model-that-outperforms",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-25T11:39:24.000Z",
  "source": {
    "name": "Hugging Face",
    "slug": "hugging-face",
    "url": "https://huggingface.co/blog/MultiverseComputingCAI/quantization-aware-healing"
  },
  "original_language": "en",
  "account": "Large language models are often made smaller by compressing the architecture and quantizing the remaining weights to 4 bits, but this process can degrade important capabilities like reasoning and code generation. To address this, researchers have developed a method called \"Quantization-Aware Healing\" (QAH) that improves the performance of compressed, 4-bit language models.\n\nQAH introduces a new approach to healing, which is the step that recovers the capabilities lost during compression and quantization. Instead of using a traditional method called quantization-aware training (QAT), which applies a noisy, lower-precision forward pass during fine-tuning, QAH distills the model directly from the original, full-precision model. This allows the smaller model to learn the output distribution of the larger, more capable model without being limited by the compression.\n\nThe QAH method involves using a frozen, pre-compression teacher model as the \"teacher\" during distillation, rather than the recovered bfloat16 checkpoint used in QAT. Since the teacher and student do not share an architecture, the size mismatch does not prevent the transfer of knowledge. The student model, now 4-bit, benefits from the teacher's full-size, full-precision output distribution, which the recovery checkpoint never had the chance to learn.\n\nIn experiments, QAH was applied to a GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4. Compared to the original 16-bit model, the QAH model achieved higher accuracy on 7 out of 9 benchmarks, while still being smaller and cheaper to run. The only benchmarks where QAH fell short were MMLU-Pro and SciCode, but the differences were less than 1.5 points.\n\nThe key advantage of QAH lies in its stability and the way it handles long contexts. By using a fixed teacher distribution through KL divergence, QAH prevents the student model from drifting during training. In contrast, QAT's cross-entropy task loss can keep pushing the student toward hard labels indefinitely, leading to instability. Additionally, QAH's memory-efficient chunked KL-divergence loss allows for healing of documents with up to 32k tokens, even on a single GPU.",
  "summary": null,
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}