Un investigador de Anthropic cree que hay más de un 10% de riesgo de que la IA "acabe con la humanidad"

Evan Hubinger, a researcher at Anthropic, a leading company in the field of artificial intelligence, has publicly expressed that there is more than a 10% chance that AI could end humanity within the next decade. Hubinger, who is responsible for alignment at Anthropic, where he oversees the development of secure and reliable models, made this alarming statement on the social media platform X. This comment came shortly after another Anthropic researcher, Jacob Coxon, announced his departure from the industry due to his belief that major labs are racing towards increasingly powerful systems without adequately addressing how to keep them in check.

Both Anthropic and OpenAI have been criticized for their approach, with Coxon accusing both companies of gambling with human lives. While Anthropic is working diligently to address the issue, Hubinger acknowledges that they do not yet have a plan to resolve alignment for superintelligence and express concern that they may not even be on the right path to achieving this goal, as the rapid advancement of AI through recursive self-improvement poses an even greater risk.

This admission of uncertainty among AI researchers is becoming increasingly common. In February, another Anthropic researcher, Mrinank Sharma, left the company after warning that the world is at risk. Samuel Marks, also an Anthropic researcher, acknowledges that many AI developers are concerned about the potential consequences of the technology they are developing, yet they continue to progress due to a combination of commercial incentives and competitive pressures.

Marks believes they are racing against less responsible rivals who would abuse the technology or develop it less securely. Both Hubinger and Marks agree that while there are techniques to push AI towards better behavior, none can robustly align them for future models. The proposed solution is to have AI systems themselves help control their even more advanced successors.

This concern led to over 1,300 employees from companies like OpenAI, Anthropic, Meta, and Google signing a statement in late July calling for international mechanisms to slow down AI development if systems begin to show uncontrollable capabilities. Proposed measures include temporary pauses in model improvements and agreements between labs to slow the pace of development.

Signatories recognize that no single company or country has the incentive to implement an emergency brake on its own, so international coordination is necessary to address this pressing issue. The academic community has been warning about the catastrophic consequences of the race toward general artificial intelligence for years.

In 2023, a manifesto called for a moratorium on AI development, warning that an uncontrolled race towards AI posed a threat to humanity. Experts acknowledge plausibility of consequences ranging from losing control over our own fate to the extinction of humanity, viewing such risks as high severity that deserves attention due to their potential impact.