Australia will impose mandatory safety rules on AI. Are we focused on the right risks?

The Australian government has recently announced six mandatory artificial intelligence (AI) safety rules aimed at regulating the most powerful AI models, known as frontier AI. These guardrails will be enforced by a new AI safety regulator, allowing for significant penalties. The six rules include implementing security controls, identifying and managing serious risks, allowing independent evaluation, rapidly notifying the government of serious incidents, publishing safety and security frameworks, and responding to updated standards.

These rules are part of a broader agenda of technology law and policy reform following an incident where an OpenAI AI agent accessed non-public Medicare statistics.

However, the focus on frontier AI models may be too narrow, as AI systems are increasingly integrated into various services, including healthcare, insurance, housing, finance, employment, education, and social security. These everyday AI systems, though not frontier models, can still cause serious harm. Examples include Robodebt, sexist AI-enabled job recruitment tools, AI-enabled stadium security systems, racist policing algorithms, and AI meal-planners that recommend poisonous recipes.

The government's approach to AI regulation appears to prioritize powerful AI models in isolation, rather than considering the broader risks associated with everyday AI systems. Moreover, AI systems are notoriously opaque, making it difficult to hold developers accountable. The lack of transparency about how non-frontier AI systems are developed and deployed hinders accountability.

The government initially proposed ten mandatory guardrails for non-frontier AI systems in high-risk settings, but these have been abandoned in favor of a lighter regulatory approach.

One promising aspect is the requirement to respond to new standards and rules, potentially incorporating detailed guidelines for fairness, transparency, and accountability. Introducing criminal liability for negligent AI developers could also encourage better record-keeping and proactive risk management. However, until the government provides more details on the scope of AI safety regulation, it remains unclear how broadly these measures will impact everyday AI systems.