The browser-training speedup was a curve, not one headline number
The WebGPU path did not have one honest speedup number. Against multithreaded WASM SIMD, the measured improvement was 2.6 times on the Small preset, 6.8 times on Medium, 9.3 times on Large, and 12.1 times on XL. Loss drift ranged from 1.1% to 2.5% across the same presets. The improvement grew with model size because larger matmuls made better use of workgroup tiling and register blocking.…
The WebGPU path did not provide a single straightforward speedup figure. Instead, the measured improvement varied significantly across different presets: 2.6 times for the Small preset, 6.8 times for the Medium preset, 9.3 times for the Large preset, and a remarkable 12.1 times for the XL preset. However, this speedup came with a trade-off - loss drift ranged from a modest 1.1% to a more significant 2.5% across these presets.
The improvement was directly linked to model size, with larger matrices capitalizing better on workgroup tiling and register blocking techniques.
The dispatch overhead proved to be more pronounced on the smaller preset, gradually diminishing in importance as the matrix work grew larger. This inconsistency in performance numbers led to the misleading nature of the single-number headline, even though the measurement was accurate. The headline had set an expectation that the improved speed could be extrapolated across all model sizes, which was not the case.
For a more accurate understanding, it is essential to consider the preset, baseline, step definition, and loss-parity results alongside the speed measurement. This context allows for a clearer picture of where the gains in performance originate. The comprehensive table detailing these findings is available on the posttrainllm.com/devlog website.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
