Apple and Meta team up to make Muse AI model run on Mac computers

Meta has introduced Muse Spark Open, an open-weight multimodal mixture-of-experts model that integrates text, images, audio, and video into a single framework. With around 816 billion parameters and a context window of up to one million tokens, this model is capable of generating text based on various input types. Muse Spark Open builds upon Muse Glimmer, incorporating a larger-scale mixture-of-experts architecture and establishing itself as the most powerful open-weight model in the United States.

Trained for agentic coding tasks, such as reading codebases, writing and debugging code, and invoking tools, Muse Spark Open can also process images, audio, and video inputs. This allows the same model to handle a variety of tasks, such as analyzing screenshots, listening to voice notes, or reviewing screen recordings while working on code. The article provides instructions on running, fine-tuning, and utilizing the model, along with the available models and their checkpoints.

Muse Spark Open is optimized for coding tasks, trained on real developer workflows like multi-file refactors and long debugging sessions. It operates as an agentic model, planning, acting, observing, and self-correcting over extended tasks, including web browsing with images. The checkpoint repositories are Muse-Spark-Open-816B-A42B and Muse-Spark-Open-816B-A42B-NVFP4.

The model's architecture includes a 62-layer language model with 8192 hidden dimensions, 128 query heads, and 16 key-value heads. Each head consists of 64 dimensions, with a separate input embedding and output layer, each having around 1.66 billion parameters. The model replaces a dense network with Mixture-of-Experts blocks, each containing 256 experts that select 8 experts per token.

This allows the model to compute tokens using approximately 42 billion parameters while keeping all 256 experts in memory for faster deployment.

The model's BF16 checkpoint is roughly 1.5 TB in safetensors format, while the NVFP4 checkpoint uses 4-bit precision for NVIDIA Blackwell GPUs, resulting in negligible accuracy loss. To deploy Muse Spark Open, refer to the llama.cpp, vLLM, and SGLang recipes. The model's perception encoder converts images and video frames into tokens, which are then processed through an audio encoder for audio input. The context window allows for 1 million tokens, including image, video, and audio inputs, at inference time.