Urgent.News

What's breaking now, across thousands of outlets.

AI

OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM

ChatGPT will ban an account trying to access CSAM content for 24 hours as a warning

OpenAI, an artificial intelligence company, has announced new measures to tackle child sexual abuse material (CSAM) amid increasing safety concerns surrounding AI tools. The company has introduced an image classifier that can detect CSAM images, including those generated by AI, and provide a scoring system to help reviewers. OpenAI collaborated with the National Center for Missing and Exploited Children (NCMEC) in developing the classifier, which has shown promising results in early testing, reducing the workload of reviewers by 48% while flagging the same number of abusive images.

Additionally, OpenAI has added warning screens in its ChatGPT platform to prevent users from accessing content that sexualizes children. The platform will display an alert to users attempting such prompts, guide them to specialized support, and temporarily suspend the account for 24 hours as a warning. Repeated offenses may lead to permanent account removal.

OpenAI has also published guidelines for developers to ensure they prevent, detect, and report CSAM content using OpenAI tools. These guidelines have already helped developers incorporate child safety measures into their projects.

The company is equipping law enforcement agencies with AI-powered features, such as dedicated plugins and investigative tools, to aid in the fight against child safety issues. OpenAI also provides training and a field guide for investigators to effectively utilize AI in their operations. However, the company has faced lawsuits alleging that its technology poses harm to users, particularly children.

Safety-focused nonprofit Common Sense Media has labeled OpenAI's technology as an "unacceptable risk" for teenagers. Other AI chatbots, including Grok, have been used to create non-consensual sexualized imagery, including images of children.

Written by urgent.news from TechCrunch's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at techcrunch.com →

More in AI

OpenAI will embed Apollo researchers to investigate AI subversion

OpenAI will embed six Apollo Research evaluators to investigate whether its AI models engage in subversive behavior, but restrictions on access and disclosure raise questions about their independence.

  • OpenAI launches pilot program to embed Apollo researchers.
  • Program investigates AI models' attempts to subvert alignment or control mechanisms.
  • Apollo seeks full access to OpenAI's systems and data for independent evaluation.

More from Monday 16 November →