Apple unleashes a swarm of Mac Minions to share GPU resources for large AI tasks

Apple has harnessed the power of its Mac minions to create a distributed virtual GPU cloud, called Minions. This innovative system is designed to make idle GPUs across its Mac devices work together on large-scale machine learning inference tasks. Prior to implementing Minions, some inference jobs faced lengthy queue times, exceeding ten hours, due to limited centralized GPU availability. However, with Minions in place, tasks can often commence within minutes by utilizing idle macOS devices.

The Mac mini and Mac Studio, recently released by Apple, support clustering using RDMA (Remote Direct Memory Access) over Thunderbolt 5. This feature enables the combination of compute and memory to support a single AI workload, allowing for the loading of models that exceed the capacity of a single machine.

Minions functions as a fault-tolerant pool of inference workers within a shared distributed inference platform. Its four primary components include the Minion Server, which serves as the coordination layer and assigns user requests as subtasks; the PostgreSQL Metadata Store, which maintains task metadata; the Minion Executor, which runs inference tasks on Mac GPUs, uploads results to S3, and reports execution status; and the Minion Watchdog, which monitors the process for failures and reassigns tasks as needed. Database statistics are fed into Grafana-based dashboards for real-time visibility.

It is important to note that Apple's Minions approach differs from Nvidia clusters, which prioritize throughput, latency, and efficient GPU utilization for large models.