Apple researchers turn idle Macs into a virtual GPU cloud with ‘Minions’

Apple researchers have developed a system called "Minions," a virtual GPU cloud composed of idle Mac computers, to efficiently manage computational workloads. Researchers detailed their findings in a recent arXiv paper. The Minions system takes on stalled AI tasks, previously waiting in long company GPU queues, and distributes them to networked Macs, reducing wait times to mere minutes. Users can initiate the process via a Slack request, though the team notes concerns about data running on devices outside one's group.

The Minions system comprises the "Minion Server," "Minion Executor," and "Minion Watchdog." Built specifically for AI inference, it utilizes Apple Silicon and encompasses two types of jobs: embeddings with Hugging Face transformer models and text generation through the Ollama platform, installed on every participating Mac. The Minion Server breaks down tasks into subtasks, which are then stored in a PostgreSQL database.

A Minion Executor on each Mac pulls subtasks via REST APIs, performs computations, and returns results to Amazon S3.

Failure detection is facilitated by the Minion Watchdog, which monitors heartbeats from Macs working on tasks every five minutes. Should a Mac enter sleep mode, disconnect from the network, shut down, or a user halts the executor, unfinished subtasks return to the queue. Teams can set priorities for job allocation, either through manual first-in, first-out or dynamic task binding preferences.

The paper suggests that each Mac executes the entire model for its assigned workload independently, contrary to splitting models across Macs. This approach mirrors volunteer computing models like SETI@home and Folding@home, where small computational tasks are distributed to idle machines. Aside from alleviating GPU queues, the Minions system eliminates the need for new GPU requests, reduces infrastructure costs to zero, and transitions scaling efforts from manual to organic. However, this does not account for electricity costs or hardware wear and tear.

Participation in the Minions system is informal; team members can request others to temporarily run the Minion Executor for specific projects or tasks via Slack messages. Confidentiality remains a concern, particularly for internal work, and Apple attempts to address this by offering team- and organization-scoped deployments with independent databases and separate Mac pools.

The development of Minions comes at a time when AI-related workloads often face compute bottlenecks. Apple acknowledged this in its July earnings report, stating that "access to sufficient computing resources" is crucial in an environment with constrained supply, prolonged lead times, and rising costs.

Apple's Private Cloud Compute, which handles more complex AI tasks for Apple Intelligence, now runs on Nvidia GPUs hosted in Google Cloud, announced during WWDC in June. This transition followed a surge in Mac demand for local AI applications, with then-CEO Tim Cook mentioning potential months-long delays in Mac mini and Mac Studio availability due to supply constraints and advanced-node challenges.

The authors of the Minions paper suggest future work could involve distributed model training and consider the potential formation of a shared GPU pool if idle Macs from various organizations, possibly even globally, can be securely federated. However, for now, the Macs in the Minions system function solely as overflow AI capacity within Apple.