AI fueles a surge in online abuse against women in India: ‘It felt like someone had taken control of how I exist visually’

On New Year's Eve, Ruchi Kokcha shared a typical selfie on social media. However, her image soon took a sinister turn as AI-generated naked versions of the photograph began circulating online without her consent. "It was digital sexual abuse," Kokcha explained to EL PAÍS in New Delhi. Using Grok, a generative AI chatbot from Elon Musk's xAI, anonymous users created fake sexual images of Kokcha.

This type of violation of basic human rights is becoming increasingly common in India's digital landscape, disproportionately affecting women. Nikhil Pahwa, founder and editor of MediaNama, a publication focused on digital policy in India, noted that the problem reflects deeper societal issues. Under India's Information Technology Rules and the proposed Digital India Act, authorities could impose heavy fines on platforms that fail to prevent the spread of harmful content.

Despite India's Ministry of Electronics and Information Technology issuing a notice to X demanding the removal of unlawful content, the damage was already done. Experts warn that platform moderators struggle to keep up with the lightning-fast pace of social media, and the legal process offers little remedy when reputations and privacy are destroyed in real-time.

Kokcha's experience coincided with the addition of new image-generation and editing features to Grok, leading to the rapid creation of sexualized images of women and minors. A recent report found that Grok generated about three million sexualized images, including 23,000 depicting children, in just 11 days. X has taken some measures, such as restricting image generation to paying subscribers and blocking certain jurisdictions, but the issue remains unresolved.

Women are now experiencing self-censorship, speaking and sharing less due to the perceived unsafe online environment.