{
  "id": 14409452,
  "title": "Ai2's Olmo-core 3 Keeps Every Expert on the GPU and Throughput Nearly Flat",
  "url": "https://urgent.news/2026/10/14/ai2s-olmo-core-3-keeps-every-expert-on-the-gpu-and-throughput-nearly",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-14T09:00:03.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/echakiri/ai2s-olmo-core-3-keeps-every-expert-on-the-gpu-and-throughput-nearly-flat-3fa5"
  },
  "original_language": "en",
  "account": "The Allen Institute for AI has released an updated open-source training framework called Olmo-core 3, which improves the efficiency and scalability of its Olmo language models. The new framework was unveiled on 1 October 2026 and is available as source code under the Apache 2.0 license. It includes technical documentation and pretraining scripts for the Olmo 3 7B and 32B models, as well as checkpoint conversion tools exporting to Hugging Face format.\n\nThe key change in Olmo-core 3 is the switch from fully sharded data parallelism to distributed data parallelism. This modification limits the need for moving stored parameters around, addressing a problem known as the \"capacity tax.\" By keeping each rank responsible for its model partition, token rows are routed to the expert holding them, resulting in higher throughput and reduced costs associated with storing parameters.\n\nFor example, a 47-billion-parameter mixture-of-experts model running on eight NVIDIA B300 GPUs achieved a throughput of about 52,000 tokens per second per GPU, compared to approximately 19,400 tokens per second per GPU with the previous implementation. This represents a roughly 2.7 times increase in throughput. Additionally, growing the expert pool from 8 to 128 experts only cost under 5% of the throughput, demonstrating the efficiency gains provided by this architecture.\n\nOlmo-core 3 rebuilds the part responsible for training mixture-of-experts models within Ai2's training framework. The release also includes optimisation techniques the team tried and did not adopt, providing other researchers with the opportunity to explore alternative approaches.",
  "summary": "TL;DR Olmo-core 3 is the open training framework the Allen Institute for AI builds its Olmo language models with, released as source code under the Apache 2.0 licence with a technical report beside it. The release appeared on 1 October 2026 and documents measured operating points from a 12.9-billion-parameter model on 16 GPUs up to a 1.2-trillion-parameter model on 512 GPUs. The central change…",
  "key_points": [
    "Olmo-core 3 replaces fully sharded data parallelism with distributed data parallelism.",
    "Throughput increases by roughly 2.7 times with Olmo-core 3 compared to previous implementation."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}