Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses

In the latest benchmarking of Qwen3.8 27B quantizations, researchers discovered that 4-bit quantization holds up well, while 1-bit model collapses completely. The full BF16 model weighs 55 GB, which exceeds most consumer hardware. However, the 17 GB Q4_K_M version matches the full model's performance on Terminal-Bench 2.1, a popular agentic coding benchmark, and fits on a 24 GB card like the RTX 4090.

Compression eventually hits a cliff at 1-bit, where the model performs around random chance on GPQA Diamond, and its reasoning gets worse with longer tasks. Previous research on Qwen3.6 27B showed good generation of SVG pelicans at 12GB and maintained knowledge up to 16GB. However, many Reddit users complained that quantizations, including 8-bit, give worse results, questioning why their local LLM feels dumber.

Measuring token prediction differences is easy, but it doesn't reveal whether the model performs worse on tasks. Focusing on benchmark results, the full BF16 model's performance was replicated, and it was found that 4-bit quantization Q4_K_M (17GB) doesn't affect results. Only the 2-bit models scored slightly lower, and other quantizations showed noticeable falls, reaching the level of Opus 4.7 or Gemini 3.1 Pro.

Smaller models need more turns, tokens or time to achieve results, and the quality drops off drastically at 1-bit. 2-bit quantizations work to some extent, but the 1-bit model is useless for these benchmarks, performing around the random guessing level. Longer reasoning further worsens the 1-bit model's performance.