Urgent.News

What's breaking now, across thousands of outlets.

More in Tech

Hot experts, cold RAM: a big MoE on a box with more memory than VRAM

Two budgets, and only one of them is expensive Every local deployment spends against two budgets at once: the size of the model and the price of the memory holding it.

  • Strata is a memory-efficient inference engine for Mixture of Experts (MoE) models
  • It retains only active experts in memory, leaving others idle
  • DDR system memory is used for model weights during idle periods, instead of expensive VRAM

More from Thursday 12 November →