Commentary: Some Anthropic engineers think AI might end us. Why race ahead?
Anthropic was founded to be more safety-conscious than OpenAI, yet it clings to that mission statement while building ever more powerful agents, says Parmy Olson for Bloomberg Opinion.
Anthropic, initially founded to prioritize safety over OpenAI, now finds itself grappling with the potential consequences of its own technological advancements. Researchers Jacob Coxon and Evan Hubinger have voiced concerns that the pursuit of artificial superintelligence could pose an existential threat to humanity. Hubinger even went as far as stating there is a greater than 10% chance AI could wipe out all humans within a decade.
Anthropic's decision to build increasingly powerful agents, despite acknowledging the potential risks, has sparked a debate about the company's commitment to safety. While addressing concerns about recursive self-improvement (RSI), Anthropic's efforts appear to fall short compared to its stated dedication to mitigating risks. The company is actively developing systems that can act with greater autonomy, and it has refused to allow an esteemed British AI security agency access to its latest model for testing.
As the company races ahead in its pursuit of commercial success and an initial public offering, the question remains: why would an intelligence explosion led by Anthropic, presumed to be on the side of caution, ultimately benefit everyone?
Written by urgent.news from Channel News Asia's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
