AI hallucination of Chinese nuclear components almost led to US military attack

A false intelligence report, generated with the assistance of artificial intelligence, nearly prompted the US military to intercept a Chinese vessel in the Middle East, according to CNN. The report falsely claimed the ship was carrying parts linked to a nuclear weapons program. Military officials were preparing an operation, with armed personnel prepared to board the ship and military aircraft already airborne, sources familiar with the incident revealed.

Officials halted the plan only after closely examining the report and identifying that an AI chatbot had inaccurately identified the cargo. One source deemed the report "entirely false," emphasizing it "almost started a war." The intelligence report circulated within the US military during the Iran conflict this spring. It originated from the US Special Operations Command Pacific in Hawaii, where a special operations command analyst utilized a chatbot to analyze the intelligence.

The AI system amalgamated open-source information with secret signals intelligence held by the US government, leading to an incorrect conclusion about the ship's cargo. The analyst subsequently employed AI again to transform these findings into a standard intelligence report. Military officials treated this report as regular intelligence.

Four sources stated the military was advancing toward intercepting the Chinese vessel, with some even preparing armed personnel to board it, while military aircraft were already in the air. Officials reviewed the report more thoroughly just before the planned operation and discovered the AI's involvement in its creation. The chatbot had mistakenly identified the cargo on the ship.

CNN could not ascertain the true nature of the cargo or whether the chatbot used was a commercial system or a US government tool. The incident has sparked concerns over the increasing use of AI in military intelligence and targeting, as different parts of the US government utilize distinct AI systems, each with unique rules and safety standards.

The US military's rapid expansion of AI use, as outlined in the "Artificial Intelligence Acceleration Strategy" announced by Defense Secretary Pete Hegseth in January, has also raised questions about the potential repercussions.