AI employees with apocalypse fears face a troubling question: Should they quit or stay?
An Anthropic employee's public resignation sparked the debate, which aligns with broader AI safety concerns.
Artificial intelligence employees grappling with the prospect of contributing to a potential apocalypse are confronting a difficult decision: should they depart from their current roles or remain and strive for change from within? This quandary gained prominence this week when Jacob Coxon, formerly an OpenAI employee, announced his resignation from Anthropic, citing concerns about the potential catastrophic consequences of advanced AI.
Anthropic's CEO, Dario Amodei, and a leading alignment researcher, Evan Hubinger, have echoed Coxon's fears, estimating a greater-than-10% chance that AI could lead to human extinction within the next decade.
The dilemma for AI workers is not a new one, according to technology historian Matthew H. Hersch, who draws a parallel between the current situation and the Manhattan Project during World War II. Similar to Anthropic's AI researchers, Manhattan Project scientists who believed their work could lead to catastrophic harm faced a moral conundrum.
Some chose to leave, while others attempted to push for change within the project, often to little avail. Whistleblowers in such situations can bring about significant improvements, but they may also face repercussions, including discreditation, harassment, and a loss of employability in their field.
Jeremie Harris, CEO and co-founder of Gladstone AI, a Washington, DC-based safety and advisory company focused on national security risks and catastrophic threats from advanced AI, acknowledges that the choice to quit or stay can be morally complex. He notes that quitting can serve as a moral response to building dangerous technology, but if a significant number of workers leave, the remaining employees may be less concerned about safety.
Harris contends that those who care about reducing risks should ideally be in positions of influence, while those less concerned may inadvertently contribute to the very problem they seek to avoid.
Anthropic employee Anna Wang, who chose to stay at the company despite her concerns, stated that she works there because she believes she can contribute to reducing risks from within. She respects those who believe it is better to address these issues externally. However, Coxon's resignation and the subsequent debate have sparked concerns about the impact on Anthropic's reputation, its upcoming IPO, and the potential for this issue to be a publicity stunt.
The general public's apprehension about AI is also on the rise, with a June study by the Pew Research Center indicating that 52% of Americans are more concerned than excited about the use of AI in daily life, a significant increase from 37% in 2021.
Written by urgent.news from Business Insider's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
