Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
The llama-modchip project enables running a 284 billion parameter Frontier Model of Expert (MoE) on a 64GB CPU workstation without any GPU hardware. This is achieved through a combination of advanced compression techniques, including Analytical SwiGLU-H2 Second-Order Hessian Compression, Fractal-64 Q2_0, and L1D/L2/L3 cache optimizations.
The tool also includes features like 4-Level Register, L2 Quadrature MoE Pruning, and zero-API Claude Code to accelerate inference speeds. The system successfully runs models like DeepSeek-V4-Flash (59GB RAM, 44.4 tokens per second) and Nemotron-3.5-30B-A3B (25-111 tok/s) while maintaining a 256K context window. This breakthrough makes large-scale AI models accessible for local, air-gapped use on standard hardware, eliminating the need for expensive multi-GPU setups.
Written by urgent.news from Hacker News's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
