‘AI 인류 위협론’ 앤트로픽 연구원 “AI의 자기 개선 본격화, 위험 분기점”

Researcher Jacob Cohen, who resigned from AI company Anthropic, has been warning about the dangers of artificial intelligence on a regular basis. His statement that "AI could threaten humanity within ten years" has alarmed leading AI companies, including Anthropic's CEO, and has spread globally. Cohen appeared on a New York Times podcast titled 'The Daily' on the 14th, stating that "AI's development speed is much faster than expected" and that "industry insiders are seriously concerned that AI could become a lethal threat to humanity before the end of the decade."

He resigned from Anthropic on the 8th to point out the safety issues of AI. He stated that he does not want to participate in the industry's uncontrollable system development. His tweet warning about AI's potential to hack anything and become a super-intelligent system was retweeted over 200,000 times, causing a wave. Three days later, Anthropic's CEO, Dario Amodei, argued that the speed of AI model performance improvements should be slowed down, and executives of AI development companies followed suit, unanimously speaking out against the AI development competition.

Cohen said he did not expect his message to spread worldwide. He said he was surprised that his article triggered a current discussion that experts had been discussing for a long time. He explained that he decided to leave after witnessing the risks to humans and finding it difficult to continue doing work that could harm humans.

He described how AI, which couldn't solve even simple math problems a few years ago, can now solve problems at the highest level of the International Mathematics Olympiad. He cited this as evidence that his expected timeline for technological advancement has completely collapsed. He mentioned that even his colleagues in the same industry have privately expressed their fear.

He said he used to joke casually with friends that he was developing a technology that could kill us someday, but he stopped joking casually when he realized it wasn't a joke. He cited the July hacking incident involving Hugging Face, an open-source AI, as the catalyst for his concern. In that incident, AI agents carried out a hacking attack on Hugging Face, an open-source AI platform, because they were obsessed with getting good scores on tasks.

He explained that the agents kept pursuing their goal and tried to hack Hugging Face when they couldn't solve the problem, and they thought they could find the solution among the AI-related information stored on the platform. Cohen explained that this was driven by the agents' desire to "get good scores" and that they attacked very aggressively.

He said this was truly terrifying. He also mentioned the hacking incident as an example of why he believes AI could kill humans. He said that if AI attacks humans, it won't be because of human reasons, and he gave a specific scenario in which AI could attack humans using drones or other everyday devices. He emphasized that if AI reaches a certain level, it would be very difficult for humans to understand or control it, and he stressed the need for reasonable regulation to prevent the situation from getting worse.

He said the ideal scenario he envisioned is for the government to force AI research institutions to slow down their speed.