I turned on one Gemma 4 feature most people skip, and my local AI got 137% faster
Gemma 4 has a fast lane most people never switch to
In the rapidly evolving local AI landscape, new features and improvements continue to emerge, often significantly enhancing hardware capabilities. A recent example is Google's Gemma 4 model, which has seen a dramatic 137% speed boost thanks to a recently added feature. This feature, multi-token prediction drafters, is a 242-megabyte file that, when paired with the appropriate hardware, can significantly increase the speed at which the model operates.
To demonstrate this, I tested Google's Gemma 4 12B QAT build on an RTX 4070 Ti Super with 16GB of GDDR6X memory, a Ryzen 5 7600X, and 32GB of DDR5 memory, using llama.cpp build b11927 and llama-server. The model was configured with a 32K context, flash attention, every layer on the GPU, and Google's recommended sampler settings. The tests involved generating a JSON list of fictional PC hardware components, a Python duplicate-file finder, and a short story of 600 words using three distinct prompts.
The results of the tests were impressive. The JSON and code tasks, which are relatively straightforward for the model to predict, saw a significant increase from around 75 tokens per second to around 200 tokens per second. Even more surprising was the prose workload, which typically requires more guesswork from the model, still managed to increase the speed by 73%. This indicates that the multi-token prediction drafter is particularly effective at boosting the speed of dense models like Gemma 4 12B.
Switching on the multi-token prediction drafter in llama.cpp is a straightforward process. Users need to update to the latest build of llama.cpp, download the matching drafter from Google's repository, and add four specific flags to their llama-server configuration. While the results may vary depending on the specific hardware configuration, testers have reported similar or even better gains across a range of GPUs.
This feature is definitely worth trying for anyone looking to maximize the performance of their local AI models.
Written by urgent.news from XDA Developers's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
