Meta ran ads for a nudify app featuring a porn video with a deepfake closely resembling a prominent US politician; Apple removed the app from the App Store (Wired)

A nudify app called Kromix, which was hosted on Apple’s App Store, featured a pornographic video of a woman who closely resembled a prominent female U.S. politician. The app promised users “no restrictions” and described it as “the AI that men actually use.” One advertisement showed the politician in front of the U.S. flag, winking before cutting to a scene of the politician in a pornographic video.

The Kromix app invited users to upload photos for AI-generated scenarios like "bedroom rape" and "Disney love." Apple struggled to keep similar tools off its App Store, receiving a cease-and-desist letter from San Francisco’s city attorney demanding the removal of 13 specific apps and stopping the sale of explicit deepfake images.

Meta ran advertisements for the Kromix app, targeting male users with the tagline "Unleash Next-gen AI Magic." The account tied to the campaign ran 32 ads, each lasting between five and 46 hours, and received between 10 or fewer impressions. Katie Paul, director of the Tech Transparency Project, expressed concerns about Meta's handling of such content, particularly its targeting of women, including politicians.

Meta's ad review system relies on automated tools to check ads against its policies, which prohibit sexual imagery. However, the company admits that it had failed to detect thousands of nudify or undressing apps. Meta has removed over 340,000 violating ads for nudify apps and launched new AI technology to better detect and block such ads. Cindy Southworth, head of women's safety at Meta, stated that the company shares the information of removed apps with other tech companies to combat this issue.