Ai2's Olmo-core 3 Keeps Every Expert on the GPU and Throughput Nearly Flat

The Allen Institute for AI has released an updated open-source training framework called Olmo-core 3, which improves the efficiency and scalability of its Olmo language models. The new framework was unveiled on 1 October 2026 and is available as source code under the Apache 2.0 license. It includes technical documentation and pretraining scripts for the Olmo 3 7B and 32B models, as well as checkpoint conversion tools exporting to Hugging Face format.

The key change in Olmo-core 3 is the switch from fully sharded data parallelism to distributed data parallelism. This modification limits the need for moving stored parameters around, addressing a problem known as the "capacity tax." By keeping each rank responsible for its model partition, token rows are routed to the expert holding them, resulting in higher throughput and reduced costs associated with storing parameters.

For example, a 47-billion-parameter mixture-of-experts model running on eight NVIDIA B300 GPUs achieved a throughput of about 52,000 tokens per second per GPU, compared to approximately 19,400 tokens per second per GPU with the previous implementation. This represents a roughly 2.7 times increase in throughput. Additionally, growing the expert pool from 8 to 128 experts only cost under 5% of the throughput, demonstrating the efficiency gains provided by this architecture.

Olmo-core 3 rebuilds the part responsible for training mixture-of-experts models within Ai2's training framework. The release also includes optimisation techniques the team tried and did not adopt, providing other researchers with the opportunity to explore alternative approaches.