Sources: US Senate negotiators are debating a bill to impose a "duty of care" for AI companies and let the government block the release of models deemed unsafe (Courtney Rozen/Reuters)

U.S. Senate negotiators are deliberating on legislation that would burden AI firms with the duty of creating secure AI products, according to multiple sources. This proposed legislation would also allow the federal government to block unsafe AI model releases in federal courts, if necessary. The objective of this "duty of care" for AI developers is to avert "catastrophic risks," according to Senate aides.

Negotiators hope to empower the U.S. government to halt the release of certain AI models deemed unsafe, but the extent of government control remains under discussion. Additionally, the bill would prevent states from enforcing their own laws concerning AI-related risks, such as the potential misuse of AI for nuclear or biological weapon development.

The proposal targets the most advanced AI models, including those developed by Alphabet's Google, Anthropic, and OpenAI. However, the bill's passage may face challenges due to the U.S. congressional calendar, with the House set to be in session only for a week before the November 3 midterm elections, and the Senate expected to be in Washington for three weeks.

Key negotiators include Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz, and top Democrat Amy Klobuchar. The Senate Commerce Committee, which oversees the Commerce Department with in-house AI safety researchers, and Senator Maria Cantwell, the top Democrat on the committee, have also contributed to the discussions.

Senator Klobuchar emphasized the need for meaningful legislation to mandate testing of the most potent AI models by scientists and experts at national laboratories to ensure they do not facilitate sophisticated cyberattacks or aid in the creation of biological or nuclear weapons.