China’s J-36 fighter jet designers report danger of military AI hallucinations
Large language models are expert in intelligence work, able to quickly mine key data and analyse weapons performance – tasks that once took analysts huge amounts of time and effort. But engineers at China’s AVIC Chengdu Aircraft Research and Design Institute, developer of the People’s Liberation Army’s backbone J-20 and next generation J-36 stealth fighters, have urged caution. In a paper…
China's top aircraft designers have issued a stark warning about the dangers posed by artificial intelligence in military applications. In a recent paper, engineers from China's AVIC Chengdu Aircraft Research and Design Institute expressed concerns about the potential for AI to generate inaccurate or fabricated information, a phenomenon they term "AI hallucinations."
These "hallucinations" can affect critical intelligence about enemy capabilities, including aspects such as radar frequencies, combat ranges, and mission plans. The engineers argue that without proper training on authoritative military data, AI systems often produce outputs that defy basic physics or operational constraints. This can lead to serious consequences in defence intelligence, potentially causing strategic miscalculations and undermining the accuracy of military assessments.
The issue has real-world implications. In 2026, a US airstrike on Iran mistakenly targeted a school, killing over 175 children, due to outdated data and an AI system that incorrectly identified it as a military target. While humans ultimately made the final decision, the speed and volume of AI-processed intelligence raise concerns about the ability of frontline analysts to verify every AI recommendation in a timely manner.
To mitigate these risks, the Chinese engineers recommend several measures. They suggest feeding AI models with trustworthy defense data, creating searchable knowledge bases for reference, writing clear and specific prompts, and using cross-checking through AI debate. Zhang Xianzhe, the lead engineer, calls for the promotion of safer, more reliable applications of large language models in defence technology intelligence.
Written by urgent.news from South China Morning Post's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
