Why Compliance Teams Keep Rejecting "Anonymized" Production Data
Why "anonymized" test data still fails compliance review: the PCI, SOC 2 and GDPR rules that apply to pre-production, and two real breaches that prove it.
During a SOC 2 audit, the auditor inquired about the source of the test data, to which the presenter explained that it was anonymized production data. However, the auditor questioned how the data was anonymized, leading to the realization that the presenter only partially understood the process. The anonymization involved replacing names and email columns with generated strings, hashing account numbers, and retaining the remaining data as it was from the nightly dump.
The presenter had been using this method for approximately eighteen months, but it took a fortnight and extensive reading to understand that the proper term was pseudonymization, not anonymization. The distinction between the two is crucial, as the classification determines whether the dataset remains personal data or not. Pseudonymization replaces identifying information with a substitute, while anonymization renders the dataset permanently unreconstructible.
The UK Information Commissioner's Office (ICO) provides guidance on this matter, emphasizing that pseudonymization merely reduces the ability to link a dataset to an individual, whereas anonymization renders the data irrecoverable. Anonymized data achieves this, while pseudonymized data does not. In a PCI DSS context, live primary account numbers (PANs) are strictly prohibited in pre-production environments unless they are part of the Cardholder Data Environment (CDE) and safeguarded accordingly.
This requirement applies regardless of the environment's name, such as staging or testing databases. The staging environment, created for reproducing customer issues, fails to bypass the stringent requirements, even with a distinct name. Both SOC 2 and PCI DSS scrutinize the origin of test data, probing whether it originates from production or not, and investigate the accessibility and duration of any such data.
Consequently, treating a staging database as confidential information strictly for internal use is misguided, as it exposes the organization to potential breaches and regulatory scrutiny, as demonstrated by the Kiddicare data breach in 2016.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
