AI could turn China’s social credit system into surveillance tool: government-funded study
China’s social credit system, stereotypically imagined as an all-powerful numerical scoring project for authoritarian control, has gained notoriety in the West – although people inside the country actually feel little of its presence or practical influence. In a government-funded study, however, a Chinese legal scholar warned that deploying artificial intelligence (AI) risked turning the rating…
A recent government-funded study warns that China's social credit system could transform into an extensive surveillance mechanism if artificial intelligence (AI) is integrated. The research, published in the Journal of Sichuan Normal University, was conducted by doctoral candidate Lin Jianxing of Sichuan University's Law School.
Lin argued that relying on big data and algorithms to gauge citizens' activities could bolster government agencies' risk assessment and regulatory capabilities. However, he expressed concern that this automated administration could lead to opacity, widespread surveillance, and discrimination. According to Lin, social rating involves using big data mining and predictive algorithms to translate people's online and offline activities into quantifiable scores.
Although a high or low score may not directly impact an individual's rights and obligations, it significantly influences their treatment in fields like healthcare, financial lending, and government services. China's social credit system, a subject of controversy, consists of various experiments run by different agencies instead of a single unified network.
The framework includes blacklists for serious legal violations, a financial credit reporting system, and municipal government-run systems that collect behavioural data and combine rewards with punishments. The State Council's AI action plan encourages the safe, steady, and orderly advancement of AI in government, aiming to create a new model of public service.
Anhui province has developed a dynamic social assistance monitoring platform using AI to enhance social services. Lin cautioned that deploying AI in the credit system may present risks, such as algorithm reliability, affected by data quality and model design. Without strict data compliance regulations and oversight mechanisms, the mechanism could transition from a risk management tool to a behavior surveillance tool.
Lin emphasized that effective operation depends on the quality of data, scoring algorithms, and standardization of administrative organs' behavior. While China's social credit system is often depicted as a uniform, threatening tool, the Stanford article highlights that only a small portion of cities in China have local systems, and the framework primarily rewards and surveils government employees over ordinary citizens, reflecting the state's aim to discipline both the public and its own agents.
Written by urgent.news from South China Morning Post's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
