Edge0-145B: Frontier AI Capabilities on Your Own Computer
Frontier AI capabilities now available on your own computer with the open-sourcing of Edge0-145B and Edge0-120B models. Edge0-145B boasts 145 billion parameters, making it the world's first open-source on-device AI model with such extensive capabilities. This release addresses constraints in four ways: providing the full stack for developers to reproduce, evaluate, and integrate the models; enabling a 145-billion-parameter model to run on a personal laptop; achieving fast token generation and low power consumption; and sustaining longer tasks.
The design focuses on reducing memory usage and data movement for large models. Architecture shares large matrices across sparse experts, training builds for local execution from the start, loading pipelines GPU memory, system memory, and SSD access, and scheduling uses CPU and GPU together for efficient decoding. Benchmarks show Edge0-145B's competitiveness with specific frontier cloud models across coding, reasoning, knowledge, and instruction following.
Rewritten by urgent.news; may contain errors — check the original.
Source: edge0.ai