The 'poison AI' movement wants to corrupt ChatGPT and Gemini to make them useless — but it comes with a huge risk of collateral damage
The movement to sabotage AI through poisoned training data may be aimed at major tech companies, but its damage could spread to ordinary people and smaller organizations.
An emerging threat in the realm of artificial intelligence is the "poison AI" movement, a group of individuals intent on corrupting AI models such as ChatGPT and Gemini. This movement aims to feed AI systems with misleading, false, or deliberately corrupted data, gradually diminishing their reliability. The underlying premise is that if AI models learn from enough inaccurate or manipulated information, they may become less dependable.
While this notion may seem plausible on paper, the potential consequences are far-reaching and potentially catastrophic.
Data poisoning is, in fact, a legitimate area of AI security research. The argument that less reliable AI systems would deter companies from using large-scale models may be enticing, yet it also poses a greater risk. By attempting to teach AI incorrect lessons, these attackers could undermine various sectors, including hospitals, banks, and government agencies, which often rely on AI systems with narrower datasets and fewer security measures.
The process of poisoning AI data involves altering vast collections of information, such as books, articles, images, and documents. Once these changes are made, they can subtly influence the AI's learning process. For instance, a model might provide incorrect answers to specific questions while maintaining overall normalcy. Moreover, attackers can embed hidden codes within images or text, which remain unnoticed by humans but can manipulate AI models when triggered.
Those behind this movement often utilize tools like Nightshade, which subtly modifies images before sharing them online. This technique makes AI systems' learning process more challenging while preserving the images' human-like appearance. However, the extent of damage caused by AI poisoning is far more severe than merely spreading misinformation. It can lead to flawed advice from medical assistants or introduce hidden security flaws in software used by critical institutions.
Despite the intentions behind this movement, poisoning AI data is unlikely to yield the desired results. AI systems are already plagued by issues like misinformation, hallucinations, and factual errors. Introducing more erroneous information into the ecosystem may exacerbate these problems further, rather than resolving them. While the concerns of artists and authors regarding AI training data misuse are valid, deliberately weakening AI models is not a viable long-term solution.
Instead, safeguarding people, their livelihoods, and creative ownership should remain the primary focus.
Written by urgent.news from TechRadar's reporting — not their text. Machine-written; read the original for the full account.
