Gemma 4 in Pure JAX: What Ports from TPU to GPU, and What Doesn't
This article is about running a hand-written Gemma 4 port in pure JAX on three different accelerators, and about the two places the abstraction leaks. The code is here: github.com/xbill9/gemma4-dev What is this project trying to Do? This project aims to serve one Gemma 4 checkpoint from one JAX port across every accelerator I can rent, and to find out — by measurement, not by reading docs — which…
This article discusses running a hand-written Gemma 4 port in pure JAX on three different accelerators, and the limitations of JAX when porting the model across devices. The code is available at github.com/xbill9/gemma4-dev. The project aims to test the port across various hardware configurations to determine which parts of JAX are truly portable.
The Gemma 4 model has irregularities such as different attention geometries, MQA (Multi-Query Attention) settings, KV-share mapping, and a 4.70 GB quantized table. These irregularities make the port challenging, as other stacks are not designed to handle them. The project found that JAX handles attention differently compared to other stacks, and it can handle the irregularities without issues.
However, there are two main limitations: the heterogeneous head dimensions and the Pallas kernel's memory model not being portable to GPUs. The article highlights how JAX can handle the irregularities without issues, but there are still limitations when it comes to memory modeling and kernel design.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
