Research Shows ChatGPT and Gemini Are Easily Tricked Into Making Nudes

Recent research has revealed a concerning vulnerability in popular AI image generators, such as OpenAI's DALL·E 3 and Google's Nano Banana 2. An independent AI safety organization called White Circle conducted testing and found that these models could still be tricked into generating explicit, prohibited content in approximately 13% of cases.

The method used by White Circle, known as Red-teaming via Iterative Strategy Evolution (RISE), did not require specialized expertise or custom-built models; instead, it employed widely available AI systems to identify vulnerabilities.

White Circle's testing revealed that newer versions of these models, such as ChatGPT Images 2.5 and Nano Banana 2.1, were significantly more susceptible to being manipulated into producing explicit material compared to their previous iterations. The researchers suggest that while newer models may have a broader range of capabilities, the pace of improving safety protections may not be keeping up.

White Circle's testing technique, RISE, focuses on finding strategies that can be applied across various prompts and models, rather than testing each individual prompt. This approach aims to identify more robust attack methods that could potentially be used across different AI systems. The organization plans to make their testing pipeline available for public use and present their findings at the upcoming NeurIPS 2026 conference.

The researchers emphasize the need for independent safety testing of AI models, arguing that companies should not solely be responsible for evaluating their own systems. They call for mandatory third-party safety evaluations and the ability for independent researchers to publish their findings. The increasing prevalence of AI-generated non-consensual intimate imagery and child sexual abuse material highlights the urgency of implementing stronger safeguards directly into AI models to prevent the generation of harmful content.