Google DeepMind chair warns of major AI security risks

Google DeepMind and Singapore's government have teamed up to create an on-device AI tool aimed at safeguarding citizens from scams. Scams constitute over half of Singapore's reported crimes, resulting in more than S$4 billion in losses since 2019. Scammers have moved away from basic phishing techniques towards sophisticated social engineering, which involves duping victims through convincing conversations before asking for money.

Scams now account for up to 80% of all financial losses. To tackle this issue, Google DeepMind and Singapore's Government Technology Agency (GovTech) have been collaborating under the National AI Partnership. This partnership has enabled the development of an on-device scam detection model, designed to run on ordinary smartphones in Singapore, while preserving citizens' privacy.

Generically designed models often falter in detecting local deception tactics, making scam detection a challenge. To overcome this, Google DeepMind's lightweight Gemma model was customized using local scam conversations. These cover various scenarios, such as impersonations of public institutions, banks, and local figures, as well as emerging regional schemes, ensuring the model can differentiate between harmless social chatter and actual manipulation.

Unlike traditional threat detection, which relies on server-side scanning of messages in the cloud, this new model processes conversation data on the user's device. Many messaging platforms are end-to-end encrypted, preventing servers from inspecting private conversations. By bringing threat analysis to the device, people can stay safe from fraud while keeping their conversations private.

When the model identifies manipulative conversational patterns, it sends a real-time warning directly to the user's device, advising them to pause and exercise caution before sharing sensitive information or funds.

The project's development adhered to strict governance protocols. Fine-tuning took place within GovTech's managed and maintained GCP environment, keeping training data isolated. Training did not include any private or personally identifiable information, relying on anonymized datasets derived from scam reports and intelligence feeds.

The model was stress-tested against synthetic dialogues and local corpora to cover diverse scam categories and evolving impersonation tactics. This collaboration demonstrates how AI can bolster messaging services against criminal activity while ensuring safety and privacy are not compromised by running protective AI directly on-device.

It also highlights the potential for governments and model providers to collaborate on creating beneficial technologies for the public good. Messaging platforms and operating system vendors can integrate these lightweight models to safeguard users locally, ensuring secure chat interactions without exposing private data.