Apple Silicon and macOS VMs: Faster LLM Inference with llama.cpp

Apple's new capability shim for macOS VMs with llama.cpp enables significantly faster Large Language Model (LLM) inference. By altering specific Metal capability responses within a single guest process, the shim allows llama.cpp to access newer GPU kernels. Tests on an M1 Ultra showed llama.cpp processing prompts 11.08 times faster and generating tokens 16.36 times faster compared to a standard macOS virtual machine (VM).

The same capability changes also improved performance for Google's Gemma 4 12B QAT Q4_0 model by 7.20 times on prompt processing and 14.54 times on token generation. With Meta's Muse Glimmer 30B Q4_K-M GGUF model, prompt processing speed increased by 7.55 times and token generation speed by 8.87 times. Apple's macOS Virtualization.framework provides a virtual graphics device that submits Metal work through a specialized GPU driver, executing it on the physical GPU via paravirtualization.

This differs from GPU passthrough in other virtualization stacks, which assigns a compatible physical PCI device or hardware function to a VM for direct access. The capability shim, inserted between an application and an API, intercepts Metal capability queries and adjusts the responses for the specific process. This change enables llama.cpp to select newer GPU paths, resulting in substantial speed improvements.

The shim is scoped to the injected guest process, leaving physical GPU assignment, raw PCI or VFIO passthrough, and kernel changes outside its scope. The findings have been documented in an open-source research release under the same permissive license as Lume and Cua.