US senators weigh requiring AI giants to commit to preventing catastrophe

U.S. Senate negotiators are deliberating over potential legislation that would compel AI companies to prove they are taking necessary measures to ensure their technology does not pose a risk, according to a Senate source and a lobbyist familiar with the discussions. Concerns about the possible dangers of AI intensified last week following the resignation of an Anthropic researcher, Jacob Coxon, who expressed his distress over the belief among AI developers that the technology could result in human extinction by the end of the decade.

Following the resignation, stock prices of AI-related companies experienced a decline globally on Monday. In response, representatives from prominent industry players have called for a temporary slowdown in the advancement of AI technology. The senators are deliberating on granting the U.S. commerce secretary the authority to request evidence demonstrating AI developers' adherence to safety protocols, referred to as a "duty of care," by the Senate aide and lobbyist.

The secretary would also possess the capacity to send government auditors to examine the products of these companies directly, the sources revealed. However, the criteria for what constitutes "reasonable" safety measures remain unclear. The legislation is still under consideration, and it faces significant hurdles in becoming law.

U.S. President Donald Trump stated on Monday that the existing regulatory and punitive powers of the government are sufficient to oversee and penalize tech companies, implying that he is unlikely to endorse a bill instituting new AI regulations. The proposal is currently being deliberated by Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz, and Senator Amy Klobuchar, the leading Democrat participating in the negotiations.

Representatives for Thune and Cruz have not yet responded to inquiries regarding the matter from Reuters. Senator Klobuchar expressed her commitment to achieving a bipartisan agreement on legislation aimed at governing the most significant risks associated with AI models, emphasizing the need for developers to collaborate with government experts to verify and test models to ensure AI safety.