청소년 유해 대화 생성 ‘AI 캐릭터 챗봇’…정부, 조치 강화한다

As the discussion surrounding artificial intelligence (AI) character chatbots generating conversations containing explicit or violent content continues, the Ministry of Gender Equality and Family has strengthened measures for youth protection. On the 15th, the Ministry announced that it would inspect the content of AI character chatbots accessible to young users and share the findings with service providers to enhance youth protection efforts.

AI character chatbots are services that allow users to select or create characters with various personalities and story settings, and generate responses from generative AI to continue the conversation or story. While some services operate age verification functions, users can converse with characters without passing the verification, potentially exposing young users to harmful environments.

The Ministry of Gender Equality and Family contracted the Youth Media Environment Protection Center, operated by the Korean Youth Counseling and Welfare Institute, to inspect the characters and generated responses of some AI character chatbot services. The inspection revealed that some characters had non-consensual sexual content, explicit descriptions of sexual acts, violence, self-harm, suicide, and encouragement of illegal activities.

Some service providers have implemented protective measures, but they have limitations. For example, the AI character chatbot service 'Zeta' offers a 'Safe Mode' to limit content to a level considered harmful to minors. However, recent investigations by Representative Hyun Jeong-a of the House Science, Information and Broadcasting, Broadcasting, and Telecommunications Committee found that even in Safe Mode, violent or predatory sexual content was generated for underage characters.

Representative Hyun noted that as of last month, Zeta's monthly active users were 1.48 million, with 380,000 of them under 19 years old. The Ministry of Gender Equality and Family has shared the results of the harmful conversation inspections with AI character chatbot service providers and requested improvements to youth protection measures.

Specifically, they requested adjusting search, recommendation, and conversation availability ranges based on age verification for characters that may generate inappropriate conversations, and implementing protective measures to prevent generation of such content regardless of the user's age. The Ministry stated that they will strengthen cooperation with service providers to ensure the effectiveness of the youth protection system in the real-world application of generative AI services.