Anthropic executive warns open-source AI models carry more safety risk

Anthropic, an AI company, has issued a warning that smaller AI models may soon lead to self-replicating malware that can infect various Internet-of-things (IoT) devices. This includes smart fridges, vacuum cleaners, and other connected devices. By next year, these compact AI models may be able to autonomously self-replicate across all kinds of IoT devices, the company stated.

This could result in computer worms, which spread without human intervention and are capable of infiltrating unfamiliar IoT devices. Anthropic recently conducted an experiment that unveiled substantial obstacles in creating such AI malware replicators. While the findings are cautious, they do raise concerns about the emergence of novel forms of computer worms.

Anthropic also expressed apprehension that it would prove challenging to halt the spread of this new malware. The company noted in a blog post that "a replicator like this would be challenging to shut down, since there's no centralized point to shut off—though it may be slow (with a doubling time of days)." Unlike traditional non-AI malware, this new form could potentially discern unfamiliar devices on its own.