Anthropic draft restricts federal powers over dangerous AI

Anthropic has shared a draft bill with US lawmakers that proposes stringent conditions for federal authorities to halt the release or operation of advanced artificial intelligence models. This legislation, disclosed on October 16th, would only allow intervention if there is a foreseeable and imminent threat related to a biological weapon, cyberattack, or an AI system that has escaped its developer's control.

The restrictions would only apply to emergency government action against these 'frontier models,' not to general definitions of acceptable harm. The difference is crucial as it means developers would still be prohibited from causing smaller losses or injuries. The draft would narrow a specific federal power to suspend development, deployment, or internal use of models deemed dangerous.

Anthropic, the company behind the Claude AI systems, acknowledges that the proposal aligns with their Advanced AI Framework, published in June. The text remains a legislative proposal, not a law, and lawmakers have not set a timeline for reviewing it. Independent evaluators registered with the US Commerce Department would periodically review frontier AI developers and their models, with the potential to notify the Commerce Department within 72 hours after identifying an imminent risk.

The department could then request a court order to suspend a model. However, other federal departments, agencies, and officers would be barred from independently ordering such suspensions. The numerical thresholds of $1 billion in damage or more than 50 deaths or serious injuries have drawn attention, as they are stricter than previous thresholds.

The proposal also limits qualifying threats to three categories and confines the special suspension mechanism to these specific dangers. Despite Anthropic's expansion of access for outside evaluators, including the research organization METR, the draft does not provide independent evaluators with the power to halt models themselves.

Instead, their findings would inform a government assessment, potentially leading to a court order if the conditions are met. This separation of responsibilities is central to the proposed enforcement design.