AI ‘암호화 데이터’ 다른 모델서 손쉽게 해독…개인정보 유출 가능성도

Recent research has revealed potential vulnerabilities in high-performance AI models, such as those developed by OpenAI, Anthropic, and Google, which could lead to the unauthorized disclosure of users' personal information. Experts warn that competitors could exploit "refinement" techniques to mimic the methodologies of other companies' models.

Researchers from the ELLIS (Ellis) Institute in Germany and the Max Planck Institute for Intelligent Systems conducted a study titled "Exfiltration of Inference Records Using Closed-Form Large Language Model (LLM) Application Program Interfaces (API)."

The study found that "encrypted inference blocks" sometimes used by modern AI models to break down problems into multiple steps before providing answers to users are often compatible across the same company's models. These "thought chains" involve solving problems step-by-step, but the intermediate results are not shared with the user, instead being encrypted as data blocks.

This encryption is done to protect sensitive user data, such as financial or personal information. However, companies have historically transmitted such encrypted blocks to users instead of storing them, leading to potential security risks.

In one experiment, researchers input the encryption block from a higher-performing model into a lower-tier model and instructed it to extract the contents, effectively "breaking out" of the safety constraints. This resulted in the decryption of both personal identifiable information (PII) and authentication details. The researchers extracted 31,532 encrypted inference blocks from 6,708 execution records of AI agents (assistants) made available on GitHub, recovering 367 PII and 182 authentication information in the process.

The researchers disclosed the vulnerability to relevant companies, which have since ceased the current attack. They recommend storing encryption blocks solely on company servers or not reusing the environment for conversations outside of the conversation sessions where the blocks were generated. The researchers also pointed out the potential for the rapid evolution of Chinese AI models, such as China's "Kimi Ke (K)3" and "Gem (GLM)-5.2" by Zhipu AI, which could adapt their response styles to resemble those of competitor models like OpenAI's "Claude Opus 4.8."

However, the researchers cautioned that they could not prove whether these models had directly copied the reasoning from the "Claude" model or simply imitated it.