How VIDRAFT Hit 510.58 TPS on Gemma-4: A Deep Dive into "The First Gemma Challenge" Win

The First Gemma Challenge was a competition focused on optimizing inference speed for the Gemma-4-E4B-it model while adhering to strict constraints, such as using a single NVIDIA A10G GPU and a limited set of parameters. VIDRAFT emerged as the winner with a TPS score of 510.58 and a PPL score of 2.3930, which passed the blind re-evaluation. This article breaks down the key configuration choices that allowed VIDRAFT to achieve this result.

Three main optimization pillars were identified:

1. Sliding-window attention narrowing: By limiting the attention window to the most recent 188 tokens, the KV-cache memory bandwidth bottleneck is reduced, and overall throughput is increased. The value of 188 was determined empirically through testing.

2. Centroid top-k kernel tuning: This parameter affects both throughput and PPL. Through sequential testing, the team found the optimal value that kept PPL within the budget. Larger values do not always lead to better performance, as they may violate the quality constraint.

3. Warm-up discipline: To account for CUDA graph capture and JIT compilation costs, the configuration includes a warm-up phase with 64 single-token dummy prompts before the timed benchmark begins. This warm-up is worth approximately 15 TPS and ensures that the measured TPS reflects the actual performance under competition conditions.

The blind re-evaluation phase played a crucial role in determining the final winner. A competing entry scored 535.91 TPS but failed the quality gate due to a PPL of approximately 2.44, which breached the 2.42 threshold. This highlights the importance of not only measuring raw throughput but also ensuring that the performance is representative of real-world, deployable inference quality.