Meta stopped worrying about distillation and just shipped the pipeline

Meta has launched Muse Glimmer, a 30-billion-parameter open-weight model, on Monday, following its distillation from Muse Spark. This open-source model is licensed under Apache 2.0 and is designed for coding, reasoning, and agentic tasks. Unlike the previous distillation method, Meta has opted to ship both the teacher (Muse Spark) and the student (Muse Glimmer) for enterprises, addressing concerns about model management.

Sam Altman had previously dismissed distillation as not being among his top ten worries, stating that capable, cheap models would exist regardless. However, Meta decided to turn distillation into a product line by using it alongside supervised fine-tuning and reinforcement learning to optimize the model for various tasks. The company owns both the teacher and the student, and intends to release the weights for Muse Spark 1.2 as well.

Glimmer is positioned outside the Frontier AI definition in Meta’s Advanced AI Scaling Framework due to its lower capability compared to Spark 1.0. The model is a targeted transfer of a subset of Spark's capabilities into something that can run on a 24GB machine. Meta has released quantization tiers for Glimmer, along with an ExecuTorch build and GGUF builds, as well as a separate DFlash speculative decoding drafter.

Unlike previous distillation practices, Glimmer is a packaged unit rather than a standalone model. Enterprises adopting this approach must track the teacher's continuity, the student's refresh cadence, the transferred capability, quantization tier, drafter pairing, and licenses for each component. Open teacher weights eliminate the need for third parties to approximate teacher logits through black-box APIs.

However, reproducing Meta's full training run remains a significant commitment of resources. As enterprises increasingly adopt this pattern, vendors may prefer labs capable of maintaining both ends of the chain on a regular basis.