Who are Jacob Coxon and Evan Hubinger? Anthropic colleagues in focus after ‘AI could kill all humans’ alarm

Jacob Coxon and Evan Hubinger, two colleagues at Anthropic, the company known for developing the AI model Claude, have expressed significant concerns about the potential dangers of artificial intelligence.

Coxon, who announced his resignation from Anthropic on September 8, stated that both Anthropic and OpenAI, the company behind ChatGPT, are racing towards developing superintelligent AI systems without sufficient regard for safety. He warned that these systems could pose a "civilizational risk" and urged caution against the current "endgame" mentality driving the development of AI.

Hubinger, a team lead at Anthropic focused on testing the alignment of Anthropic's models, echoed Coxon's concerns. He acknowledged that AI could potentially kill all humans, expressing a personal belief that there is a 10% chance this could happen within the next decade. While Anthropic is actively working on alignment strategies, Hubinger emphasized that they are not yet on track to solve the alignment problem for superintelligent AI.

Hubinger highlighted that the risk from current AI models is relatively low, but the potential danger of superintelligence arising from recursive self-improvement is a significant concern. He noted that Anthropic's alignment strategies are still under development and that the company's rapid progress in AI capabilities could outpace their ability to address safety issues.

The pair's statements come as part of a broader debate within the AI community about the pace of development and the need for more rigorous safety measures. Coxon called for a temporary ban on improving model capabilities until a more comprehensive understanding of the risks is achieved, arguing that attempting to "speedrun alignment" from a private company's Slack is a hubristic gamble.