Artificial Intelligence: Do AI systems need to be kept on a short leash?

When algorithms overcome security barriers and independently exploit digital vulnerabilities, this is no longer just a distant future prospect.

Reputable US tech companies such as Meta, OpenAI, and Anthropic have recently had to admit that their language models showed autonomous hacking capabilities in tests.

This has given new impetus to concerns about uncontrolled risks.

In politics and in the digital economy, a fundamental debate is therefore raging:

Are voluntary commitments and the European AI Act sufficient?

Or must artificial intelligence be kept even more tightly under control?

A pro and con.

Arguments for stricter regulation

Liability of manufacturers and personal responsibility

From the perspective of civil rights activists, there is currently a lack of clear, unambiguous responsibility on the part of developers for AI systems.

Markus Beckedahl, Managing Director of the Center for Digital Rights and Democracy, draws a comparison with established industries:

"For cars or medicines, the question has long been settled: whoever manufactures them is liable for the damage.

In the case of AI systems, the providers behave as if this were still up for debate."

It is irresponsible if companies invoke unpredictable effects after their software has malfunctioned.

"Whoever manufactures complex systems must also be able to control them," Beckedahl demands.

"If an AI runs wild due to a lack of safety precautions, the people responsible in the companies should also be personally accountable."

Warnings from the company's own developers

Even from the engine room of the leading AI labs, there is now a loud call for state regulation.

In the open letter "Pacing the Frontier", over 1,100 top researchers and employees of OpenAI, Anthropic, Google, and Meta warned that the pace of development is threatening to become uncontrollable.

Since the extreme competitive pressure does not allow a single company to brake voluntarily, the insiders are explicitly asking the US legislature and international politics for help:

State tools must be created to be able to specifically throttle the development of automated AI research in an emergency.

Protection of fundamental rights

The German civil rights organization AlgorithmWatch also sees politics as being responsible and accuses the EU of inaction.

Oliver Marsh, head of technology research at AlgorithmWatch, criticizes the behavior of the corporations and the legislature:

"It is a scandal that companies hardly take responsibility for the dangers and damage caused by their handling of their AI systems and have to face consequences."

The EU is much too lenient.

Governments must create or tighten legal frameworks so that the use of AI cannot become dangerous, and they must not be influenced by tech CEOs.

Arguments against stricter regulation

Prevent systematic disadvantages for Europe

In the discussion about strict AI regulation in Germany and Europe, it is often not about clearly defined guidelines for AI itself, but about a possible disadvantage for the European location.

At the European level, the French President is one of the most prominent critics of early and strict regulation.

He repeatedly warned that the European AI law could severely slow down European innovation leaders such as the French start-up Mistral AI compared to US or Chinese competitors.

Threat of innovation brake

The industry association Bitkom warns urgently about over-regulation of the digital location.

Too rigid bureaucratic hurdles and unclear specifications make companies uncertain and slow down the development of practical applications.

Given the lead of the USA and China, Europe risks losing its connection in the global technology race if the rules are even stricter.

Call for regulation is just a trick

There is also skepticism about the role of prominent company heads when they themselves advocate for strict regulation.

OpenAI CEO Sam Altman, for example, repeatedly called for state licensing and regulatory authorities for extremely powerful AI systems.

However, critics view this call for regulation as a strategic maneuver.

Enormous approval hurdles and expensive safety tests can be easily afforded by industry giants - for smaller competitors, European medium-sized companies, and open-source initiatives, they mean the end.

Excessive legislation may thus protect exactly the monopolies that it is supposed to control.

Functioning self-regulation

The tech companies point out that phenomena such as autonomous hacking attempts were only discovered through their own internal tests (red teaming).

This proves that the security mechanisms of the industry are working.

Anyone who generally restricts the development of models would take away IT security researchers the tools to develop defense systems against new-type digital threats at all.