The Right to Be Forgotten Is Hard for AI: Why Deleting Your Data From a Model Isn’t a Delete Button
You ask a company to delete your data. In a normal system that is a database operation: find the rows that are yours, remove them, done. The mental model of “delete” that privacy law is built on — the GDPR’s right to erasure, most obviously — assumes exactly this: that your data sits somewhere as a discrete record you can locate and destroy. A trained AI model breaks that assumption. Answer…
When you request a company to delete your data, they are essentially attempting a database operation: locating the relevant rows and removing them. However, this mental model doesn't apply to AI models. Your data isn't stored as individual records, but rather dissolved into the model's parameters, which are billions of numbers adjusted during training with every example.
There's no single row with your name to delete. Instead, erasing your influence requires changing these numbers, a complex research problem rather than a simple toggle. The process of training a large AI model involves adjusting its parameters based on prediction errors, repeating this across vast amounts of data. The crucial point is that this adjustment is distributed across many parameters, making it hard to pinpoint the exact area impacted by your data.
Deleting your original document from the training data doesn't affect the model that was already trained on it. In essence, deleting your data from a dataset is akin to removing a lump of sugar from a cake that has already been baked - the lump is gone from the recipe, but the sweetness remains in the cake. The ideal solution would be to retrain the model from scratch, excluding your data, a process known as exact unlearning.
However, this is often impractical due to the high costs associated with training a large model. A more affordable approach is approximate unlearning, which tries to make the model act as if it never saw specific data without the expense of full retraining. Techniques like gradient ascent, influence functions, and sharded retraining aim to bring down the costs, but they come with trade-offs, such as potential degradation in model performance.
The methods are still being researched and tested, and their effectiveness is often measured through empirical tests rather than theoretical proofs. Despite these advancements, the fact remains that even the most thorough erasure methods leave some traces that could potentially be recovered. Therefore, when a service promises to "delete your data", it usually refers to the training set and the databases, not the model that has learned from your data.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
