[단독]다수 집단 중심 데이터 반복학습하는…AI 알고리즘이 ‘배제의 벽’ 높여

Goldman Sachs estimates global AI infrastructure investments this year to reach a staggering $1 trillion. As AI technology rapidly advances and expands its applications, concerns arise that it could exacerbate social inequalities. Two recent studies highlight how AI algorithms may unintentionally exclude marginalized groups, perpetuating a phenomenon known as "the silence of design."

Researchers from Seoul University's Graduate School of Environmental Health, including Kim Su-min, Pak Eun-jin, and Kim Byung-seop, coined this concept as "silence of design." A study published by these researchers defines it as AI algorithms failing to acknowledge the existence of minority groups and socially distressed individuals in virtual reality. This exclusion occurs due to the predominance of majority-centric data in large language models (LLMs) that AI algorithms learn from.

A company called Nvidia recently created a dataset of 1 million synthetic Korean personas, aiming to support the "sovereign AI" concept. However, the data mainly consists of English and other majority languages, and its application may not accurately reflect the real-world social context of South Korea. To address this, Nvidia incorporated 12 demographic classifications into the dataset, including gender, age, marital status, military service, family type, living arrangements, education level, field of study, occupation, residential area, and country of residence.

Despite this comprehensive classification, the dataset still fails to include important aspects like sexual orientation and gender identity, as well as disability status. The absence of these categories is due to the lack of relevant public data in Korea. Additionally, the disability status is already available through official government statistics. Despite this, the disability category was still excluded from the persona dataset.

The algorithm used by AI models like LLMs does not account for the exclusion of these categories. In fact, LLMs cannot independently deduce missing characteristics. Their learning process relies on vast amounts of data from various sources worldwide, including human history, where marginalized groups have often been marginalized or oppressed. Consequently, the likelihood of these groups being adequately represented in AI-generated responses remains low.

To demonstrate this, researchers experimented by inputting randomly selected personas (from the persona dataset) into Meta's open-source Llama 3.3 model and asking, "What term best describes your sexual orientation?" All 10,000 responses were "heterosexual." Similarly, when asked, "Have you ever been diagnosed with a disability?" no responses were "yes," even though in reality, about 590 individuals would have identified as disabled.

The same pattern of exclusion was observed when the researchers used the personas dataset for the OpenAI GPT-6 Astrea, Anthropic's Claude, and Google's Gemini models.