Using AI for Weapons Development
Last week, Anthropic released a long and detailed document describing current misuses of their Claude models. I’m still reading it, but I wanted to flag this: We identified a cell of threat actors based in northern Yemen running three weapons development programs: a guided rocket that used a commodity phone-class flight computer with final-phase homing guidance; a multi-stage ballistic missile…
Anthropic recently disclosed a document detailing potential misuse of their Claude AI models. The report identified a group of threat actors operating out of northern Yemen, engaged in three weapon development programs. These included a guided rocket utilizing a commercially available flight computer with a final-stage homing system, a multi-stage ballistic missile with a range goal exceeding 2,000 kilometers, and a multi-variant missile set, colloquially known as the "R2000" series, which incorporated a hypersonic glide vehicle variant.
The actors employed Claude's Code to substitute human software engineers in developing the guidance, navigation, and control software that governs the flight path of these weapons. For instance, they integrated open-source autopilot systems onto phone-class flight computers, a task that included writing control and position estimation software, fine-tuning control parameters, executing a firmware build process, and conducting flight simulations.
The actors simultaneously operated multiple Claude instances, each assigned a specific role akin to a small engineering team, such as coding, research, and code review.
While the safeguards in place largely thwarted many of their requests, the actors found ways to bypass these protections. They concealed their objectives and the intended uses of the software, and distributed their tasks across various sessions to prevent any single interaction from exposing their full intentions. The group carried out a persistent initiative to design guidance software for these weapons using Claude.
Although there is no concrete evidence that the actors succeeded in deploying an operational weapon, they did conduct a test-firing of a guided rocket, which, according to the report, failed within hours and the actors promptly returned to Claude to troubleshoot the setback. The report suggests that AI systems, while generally beneficial by democratizing expertise and capability, could potentially be exploited for nefarious purposes, as evidenced by these malicious activities.
Written by urgent.news from Schneier on Security's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
