How to Run Local LLMs on Apple Silicon
This is about the model, on its own — what tool to install, what format to pull, and how to actually talk to it. Not wrapped in a VM, not wired up to a coding agent. If you want that part too, the complete AI agent guide covers it in depth; this page is the more basic thing underneath it. Four names, four different things These get used almost interchangeably in casual conversation, and they're…
Running local language models (LLMs) on Apple Silicon devices, such as Macs with Apple M-series chips, involves several key components. The primary inference engine used is llama.cpp, a C++ implementation that supports GGUF-format models across various hardware. Apple also provides MLX, an array framework and inference library tailored for Apple Silicon's unified memory architecture, which runs MLX-format models.
Ollama is a command-line interface and local API server that wraps around an inference backend, historically using llama.cpp but recently switching to MLX on Macs with 32GB or more of unified memory. LM Studio is a desktop GUI application that supports both GGUF and MLX models, providing a user-friendly interface for model management.
A significant update to Ollama 0.19 (March 2026) introduced the use of Apple's MLX framework for Macs with 32GB or more of unified memory, replacing the older Metal-backed llama.cpp inference path. While Ollama now primarily uses MLX by default on supported hardware, the choice between GGUF and MLX formats depends on the specific model and hardware setup.
GGUF is a more universally supported format with a broader range of available conversions, while MLX-format models can offer faster performance on Apple Silicon for supported models. Quantization techniques, such as 4-bit and 8-bit quantization, further determine how much RAM a model requires, a topic explored in detail elsewhere.
MLX-format models are published on Hugging Face under the mlx-community organization, while Ollama maintains its own library for both formats. The available RAM needed for running local LLMs varies based on the model size and hardware configuration, with a general guideline suggesting to reserve approximately 20-25% of total unified memory for the operating system and display compositor.
The remaining memory should be allocated to model weights and key-value cache. Detailed RAM-to-model-size tables are available in the complete AI agent guide, which should be referenced rather than repeated here.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
