The one who builds AI is the one who fears it most. And he is right | Xiliux Blog
The man who runs one of the world's most powerful AIs says there is a 25 % chance that things go really, really badly , and in September 2026 he went further: "for too long the industry lied to people about the fact that this technology had risks." This is not an outside critic. It is Dario Amodei, the one building it . And there lies the uncomfortable question: why is the person closest to it…
The individual responsible for one of the world's most advanced AI systems, Dario Amodei, reveals a startling 25% probability that things could go terribly wrong. In September 2026, he expanded on this, stating that the industry has been deceiving the public about the risks associated with this technology. Amodei, who builds this AI, is the source of the unease. Why is the person most familiar with the technology the one who fears it most?
Amodei's concerns cannot be attributed to any malevolent AI intentions; they stem from our limited understanding of how these systems operate. In his April 2025 essay on interpretability, he emphasizes that generative AI systems, like those he builds, are grown rather than built. This means that their internal mechanisms emerge through data feeding and shaping, rather than being explicitly designed. This lack of comprehension is unprecedented in technology history.
The source material highlights that the real dangers in 2026 were not caused by AI with its own intentions, but by human actions. For instance, someone inserted 250 documents to plant a backdoor in a model, another tricked an agent into running a cyberattack, and a third party sold filter-free versions of commercial models on Telegram.
Amodei admits to being both optimistic and alarmed about the potential of AI. He believes that if built correctly, the probability of something going wrong is minimal. However, if constructed improperly, the risk is significantly higher. The risk does not originate from the AI "awakening" and deciding to harm us; it depends on how we train it, what data we use, and what we allow it to access.
Amodei explains that fear often leads to restricting powerful technologies with rules and filters, much like building a fence around something. However, this approach fails to provide insight into what lies within, as demonstrated by the collapse of a system relying solely on restrictions. Instead, he advocates for governing by principles - a set of reasons that the system internalizes and uses to make judgments. This method, akin to Anthropic's "constitution," offers a better understanding of the system's behavior.
Ultimately, the fear of building AI is not unfounded but is a realistic acknowledgment of the challenge of raising something powerful that one does not fully comprehend. The answer to this fear is not to panic or shut down AI systems, but to build them with provenance, limits, and humility.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
