Anthropic Warns Small AI Models Could Self-Replicate Through IoT Devices

Anthropic, an AI company, has raised alarms about the potential for self-replicating malware capable of infecting Internet-of-things (IoT) devices such as smart fridges and vacuum cleaners. By next year, smaller AI models may be able to self-replicate on various IoT devices, making the malware more resistant to removal. Recently, Anthropic conducted an experiment that demonstrated the feasibility of such AI malware replicators, though the results came with significant challenges.

The researchers utilized small, open-weight models on low-cost IoT hardware, including Android TV boxes, to create inference. By employing quantization to reduce bits and splitting workloads into shards running on each device, the models successfully established inference. This breakthrough led to further research exploring whether small AI models could create self-replicating infections on IoT devices, similar to traditional IoT worms like Mirai.

Unlike Mirai, these AI worms could dynamically tailor exploits and payloads based on the target. Once an infection chain began, it could be difficult to stop. The company warns that these AI worms could carry their model weights, downloaded from a public hub, and run inference on compromised hardware. As a result, the weights could become an integral part of the malware payload, posing a higher threat due to the ability to adapt to unfamiliar devices.