US Military had close call after using AI for hallucinated intelligence report

A US military intelligence report, distributed across the US military in the midst of the conflict with Iran, sparked alarm: a Chinese vessel in the Middle East was purportedly carrying components of a nuclear weapons program. Military forces were prepared to intercept the ship, with armed personnel readying to board and planes in the air, according to four sources familiar with the incident.

It was only moments before the operation was set to begin that officials discovered the report had been generated by artificial intelligence (AI), a chatbot used by the analyst had incorrectly identified the cargo on the vessel. CNN was unable to ascertain the nature of the misidentified cargo. The report, according to one source, was entirely false, but the source added that it "almost started a war."

The use of AI in targeting military assets has raised significant concerns, as the incident demonstrated the potential for catastrophic miscalculations when relying on flawed or inaccurate data. The episode emphasizes the need for robust safeguards and human oversight when employing AI systems in critical decision-making processes within the military and intelligence community.