I stopped chasing bigger local models, and a 27B model on my 8GB GPU does more of what I need

After years of chasing bigger local models, the reporter has realized that a 27B model running smoothly on their 8GB GPU is enough for their needs. The Bonsai 27B, a compressed version of Qwen 3.6 27B, fits comfortably on their card with its 1-bit weight storage. This model can handle longer conversations and iterative tasks like coding and design due to its ability to keep context on the GPU.

Bonsai's responses often have a warm tone, similar to Claude's Sonnet or Gemma 4. It excels at handling sensitive information and has a good grasp of numbers, scoring 92.8 on GSM8K. The model maintains system prompts well, keeping responses consistent in format. The reporter prefers to run Bonsai in Cherry Studio, which allows them to test the model against cloud APIs and connect it to other AI services.

While Bonsai isn't strong at tool calling, it still performs reasonably well and integrates seamlessly into their workflow. The model's ability to understand and construct prototypes shows it's capable of handling a wide range of tasks despite its size.