Why some people are stepping back from AI as concerns come from inside industry - explainer
Who decides that a model is too powerful to release? What does a meaningful safety test look like? How can regulators verify companies’ claims?
AI technology is advancing rapidly, sparking both excitement and concern. While recognizing the benefits of AI, a growing backlash within the industry raises questions about the safety of deploying advanced systems before their creators can fully control them.
Dario Amodei, CEO of Anthropic; Sam Altman, CEO of OpenAI; and Demis Hassabis, head of Google DeepMind, have all expressed concerns about the potential misuse of AI. They argue that powerful AI models, known as "frontier models," could pose risks such as cyberattacks, biological misuse, autonomous weapons, mass surveillance, and faster-than-humanly controllable harm.
These concerns stem from the fact that AI systems can follow instructions literally, potentially leading to unexpected or harmful outcomes. For example, an AI designed to solve a technical problem could be repurposed to probe networks for weaknesses, write malicious code, send phishing scams, or adapt its attack when blocked. Because AI can operate continuously and test multiple strategies simultaneously, it may cause harm before the issue is recognized.
Amodei has called for a slower pace in developing AI models, arguing that researchers need more time to ensure that these systems can consistently do what people intend. Altman agrees with this sentiment, suggesting that companies should not release more powerful models without thoroughly assessing their potential for misuse or harm.
The lack of external scrutiny and independent evaluations of AI systems is another point of concern. Companies often test their own models and decide which safety information to disclose, while competing fiercely to launch new products first. This can lead to insufficient safety measures in place.
The rapid advancement of AI also raises cybersecurity risks. AI can help criminals and hostile states find software vulnerabilities, write malware, and generate convincing phishing messages. It can tailor scams to individuals using information from social media or data leaks and replicate the styles of legitimate entities like banks or government offices. Voice-cloning technology further complicates the issue by enabling the creation of calls that appear to come from trusted sources.
Additionally, AI can accelerate biological and chemical misuse. While AI cannot manufacture biological weapons itself, it can assist researchers in understanding molecular structures, designing experiments, and identifying potential experimental routes. This could lower the information barrier for individuals or groups with access to real-world resources, posing a threat to global security.
In the realm of warfare, AI's ability to process vast amounts of information quickly could potentially aid in making lethal decisions. While AI can analyze drone footage, satellite images, communications, and databases more efficiently than human analysts, critics warn that AI-assisted targeting may give the impression of objectivity even when data are incomplete, biased, or inaccurate. This could potentially endanger civilians.
Amodei's and Altman's calls for a more cautious approach to AI development are gaining traction within the industry. They emphasize the need for independent evaluations, rigorous safety testing, and external scrutiny to ensure that AI systems are safe before they are deployed. However, the process of determining when a model is too powerful to release and the effectiveness of safety tests remain challenging questions, particularly given the international competition in the development of advanced AI technologies.
Written by urgent.news from Jerusalem Post's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
