Urgent.News

What's breaking now, across thousands of outlets.

AI

OpenAI introduces new image classifier, warning screens to tackle CSAM on ChatGPT

OpenAI introduces new image classifier, warning screens to tackle CSAM on ChatGPT

OpenAI has introduced new measures to combat child sexual abuse material (CSAM) on its ChatGPT platform, aiming to address growing concerns about the potential risks AI chatbots pose to children. On Monday, the company unveiled an AI-powered image classifier capable of detecting CSAM images, including those generated using artificial intelligence.

The classifier, developed in collaboration with the National Center for Missing and Exploited Children (NCMEC), scans images for visual signs of abuse and assigns a score to aid CSAM reviewers in assessing the content. During testing, the classifier flagged 48% fewer images but detected the same number of abusive images, with flagged images being twice as likely to be CSAM, thereby reducing the workload for human reviewers.

In addition to the image classifier, OpenAI announced the addition of warning screens within ChatGPT. These screens will appear when users attempt to generate CSAM or related content, providing alerts and directing users to specialized support aimed at stopping harmful behavior. Accounts of users who repeatedly violate these rules will be permanently banned. OpenAI maintains its existing policy against generating CSAM content and blocking related user requests.

Furthermore, OpenAI released guidelines for developers to help them implement child safety tooling and report CSAM to the NCMEC. The company also introduced tools for law enforcement agencies, including AI-powered child safety features such as dedicated plug-ins, investigative tools, and sophisticated data analysis. Access to these law enforcement tools is managed by program administrators, and investigators will receive training sessions along with a reference guide on utilizing AI for safety purposes.

Written by urgent.news from The Indian Express's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at indianexpress.com →

More in AI

More from Tuesday 17 November →