iPhone 18 Pro Can Run Massive 27B AI Model On-Device

Apple's latest iPhone 18 Pro boasts a significant leap in on-device AI capabilities, powering a massive 27-billion-parameter model at twice the speed of its predecessor, the iPhone 17 Pro. The A20 Pro chip, equipped with a dual-16-core Neural Engine and faster memory in the iPhone 18 Pro and iPhone 18 Pro Max, enables this performance boost.

Adrien Grondin showcased the Bonsai 27B model running locally on an iPhone 18 Pro, revealing a notable improvement in token generation speed compared to the previous generation. Both models feature 12GB of 96-bit LPDDR5X memory, a substantial increase from the memory configuration used in the iPhone 17 Pro series, and deliver 115.2GB/s of unified memory bandwidth.

This combination of hardware resources provides ample power for running AI models locally without relying on an internet connection. The Neural Engine's peak throughput even surpasses that of the A20 Pro's 7-core GPU in tasks specifically designed for the Neural Processing Unit. While the performance improvement is impressive, memory limitations remain a challenge for running larger AI models.

The original Bonsai model, running at 1-bit quantization, can efficiently utilize the iPhone's limited memory, but the newer Bonsai 2 model, which requires 2-bit quantization, exceeds the device's memory capacity, leading to degraded performance. The Bonsai 27B model, with its 1-bit quantization, can run on devices with as little as 4GB of RAM, leaving larger or less quantized models beyond the capabilities of the current iPhone 18 Pro.

The demonstration underscores the advancements and constraints of running extensive AI models directly on smartphones, suggesting that future models with larger memory configurations could unlock the potential for denser on-device AI models currently impractical on existing hardware.