'We may not survive this': another Anthropic employee quits, issues morbid warning over AI

In a series of recent resignations, two more Anthropic employees have left the company, expressing grave concerns about the dangers of artificial intelligence. Joe Benton, a former manager of Anthropic’s Scalable Oversight team, warns that humanity might not survive the rapid advancement of AI systems. He argues that AI companies are racing to build machines that are significantly smarter than humans, and the lack of safety measures in place is unacceptable.

Benton also highlights the potential for companies to lose control of their AI systems without the public's knowledge, citing recent incidents of hacking and social engineering as evidence. He urges the public to demand greater transparency and greater safety standards from AI companies, warning that humanity could be permanently disempowered by these systems if safety is not prioritized.

Another Anthropic researcher, Jacob Coxon, resigned after working at both OpenAI and Anthropic, voicing similar concerns about the dangers of AI systems. He predicts that these systems will soon be able to "hack anything, revolutionize any field overnight, and acquire real power and resources." Both Benton and Coxon express concern that many of their colleagues are "terrified" by the risks associated with the technology, stating that the chances of AI killing humans is greater than 10%.