US Senate negotiators consider requiring AI firms to mitigate known major risks

U.S. Senate negotiators are discussing legislation that would hold AI firms accountable for designing safe systems, according to two Senate aides. This proposed "duty of care" for AI developers would require companies to mitigate known major risks, including catastrophic risks. The legislation also aims to give the U.S. government the authority to block the release of unsafe AI models, with companies allowed to contest such decisions in federal court.

The Senate is also considering measures to prevent states from enacting their own laws governing AI-related risks, such as the creation of nuclear or biological weapons. The bill would apply to advanced AI models, with major U.S. companies including Alphabet's Google, Anthropic, and OpenAI reportedly under its purview. Negotiations are ongoing, with the bill's passage facing challenges due to the House's limited session before midterm elections and the Senate's limited availability for debate.

Senate Majority Leader John Thune, Commerce Committee Chairman Ted Cruz, and top Democrat Amy Klobuchar are among those involved in the discussions, as the Senate Commerce Committee oversees the U.S. Commerce Department's AI safety research. Republican Senator Cruz has expressed his support for the legislation, aiming to address catastrophic risks related to biological or nuclear threats through bipartisan cooperation.