AI and the imitation game

In a concerning development, artificial intelligence (AI) technology is being used to clone voices, raising alarms about potential scams and exploitation, particularly targeting vulnerable groups such as the elderly and digitally illiterate individuals. Maria, a 60-year-old woman, experienced this firsthand when she received an urgent request from her sister, Fauzia, to wire her 50,000 rupees, only to discover later that Fauzia had never called her. The scam artist utilized AI voice cloning to impersonate Fauzia's voice.

Instances of Microsoft's Copilot, an AI assistant, mimicking user voices have been reported, and a woman in the U.S. lost thousands of dollars after receiving a phone call with what she believed was her daughter's voice. The issue is further exacerbated by apps like ElevenLabs, which offer instant voice cloning for a monthly fee, allowing users to clone a 10-second audio clip of someone's voice.

International celebrities have also fallen victim to unauthorized voice cloning, with a UK-based voice-over artist's voice being downloaded over 900 times by an American AI firm without her consent. This highlights the lack of proper legislation and control in developed countries, which could provide valuable lessons for Pakistan.

The elderly population in Pakistan is particularly vulnerable, with an estimated 6-7% of patients living alone due to brain drain. They are often targeted by criminals as they may be unfamiliar with AI-generated voices, socially isolated, and prone to trust. Additionally, digitally illiterate individuals, such as those who rely on digital banking or frequent WhatsApp usage, are also at risk.

Voice authentication is crucial for verifying identity and discussing financial matters, but AI-generated voices can easily deceive this process. As generative AI produces highly convincing images, videos, and synthetic content, even personal characteristics like handwriting and writing style can be replicated.

The ultimate concern is that AI could render traditional identity markers, such as face, voice, handwriting, words, and mannerisms, reproducible. This raises the question of what remains to prove one's identity. It is crucial for the Pakistani government to recognize the significant scope for abuse and exploitation and prioritize stronger legislation, along with educating the public about the risks associated with AI.