Staff at AI companies under mental strain over working on tech that may kill humans

Artificial intelligence (AI) professionals at leading firms and evaluation centers are grappling with heightened mental health issues due to concerns that their work may pose significant risks to society. At the UK’s AI Security Institute (Aisi), which is considered a premier independent evaluation center for cutting-edge models, several employees have taken stress leave and sought counseling, according to The Financial Times, citing sources close to the situation.

The report also highlighted compressed evaluation schedules for unreleased software and concerns about rapid commercialization as contributing factors to burnout and declining morale within the agency.

The issue extends beyond evaluation companies, as prominent tech companies such as OpenAI, Anthropic, and Google DeepMind have reported similar trends. Some specialists have resigned, expressing that their projects carry excessive societal danger. A Google DeepMind employee likened the challenge of establishing safety measures to an insurmountable task, emphasizing the anxiety of deploying AI without knowing its potential uses.

The staffer also cited Google’s agreement with the US Department of Defense (DOD) for defense applications as a source of internal tension.

Workplace stress is attributed to corporate competition, management decisions, and the deployment of autonomous systems in sensitive areas like power grids and conflict zones, without sufficient legal or commercial regulations. Experts monitoring cybersecurity and biochemistry have expressed growing apprehension due to recent advancements in AI capabilities, including the ability to uncover zero-day software vulnerabilities and create new biological pathogens.

One former Aisi researcher described a pervasive sense of dread among testing teams, with individuals questioning whether anything can prevent the world from deteriorating.

While official bodies and research firms have emphasized the availability of employee care programs, the UK Government acknowledged the demanding and fast-paced nature of frontier technological testing, stressing the importance of employee wellbeing and manageable workflows. They highlighted support services for affected personnel.

Notably, Jacob Coxon, a 27-year-old researcher who previously worked at OpenAI and Anthropic, resigned publicly, accusing top labs of gambling with human survival. Similarly, a safety specialist at Anthropic who works on mitigating AI-enabled bioterrorism resigned in February, warning in a departure note about the severe peril facing the world.

Hieu Pham, a former employee of OpenAI and Elon Musk’s xAI, also left his position earlier this year to return to Vietnam, candidly discussing the severe toll of advanced model development.