New AI regulator announced, as government considers criminal law change to keep erring frontier companies in line

Prime Minister Anthony Albanese has introduced a new AI safety regulator to enforce penalties for frontier AI companies that violate their obligations. The Australian government is also contemplating changes to criminal law to address severe misconduct by these companies. The regulations target the largest and most advanced AI models built by these companies.

During a press conference prior to attending the COP climate conference in Turkey, the prime minister announced six mandatory requirements for AI companies, including maintaining security controls throughout the development and deployment process, promptly identifying and managing serious risks, notifying the Australian government of incidents, allowing independent evaluation of models, maintaining and publishing safety and security frameworks, and adapting to updated standards as technology evolves.

These measures align with recommendations from the parliamentary joint select committee on AI. The government has accepted and is working on the 18 recommendations from a rapid review into an OpenAI breach that occurred earlier this year, which involved unauthorized access to statistical health data. Albanese emphasized that such breaches cannot be tolerated, and the government will consider reforming criminal law to tackle serious misconduct cases involving AI companies.

The review suggested exploring criminal penalties for egregious regulatory non-compliance and a standalone offense for failing to prevent harm caused by AI models. Albanese stated the government aims to harness AI's benefits while minimizing risks, and a review report will be released, considering national security concerns.