Block's new voice AI sounds eerily human, sparking awe and scam fears

Block, the parent company of payments giant Square and digital currency wallet Cash App, has introduced a new voice artificial intelligence (AI) model named Dessa. According to company announcements, Dessa is designed to sound remarkably human when conversing with users. Launched as a research preview, the technology was trained entirely on human speech rather than text and boasts the ability to listen while speaking, enabling it to effectively handle interruptions, laughter, and fluctuations in tone during conversations.

In demonstrations, Dessa reacted naturally to various situations, such as laughing mid-sentence and expressing what sounded like genuine emotion in response to surprising statements. Jack Dorsey, CEO of Block, expressed amazement at the model's conversational abilities, stating that interacting with Dessa felt like speaking with a real person, complete with emotion, nuance, pauses, and a natural cadence.

The AI model was trained using a single Nvidia GB200 NVL72 computing system and can operate locally on a MacBook Pro equipped with an M4 Max chip or higher. While Dessa remains a research preview, its potential applications include education, where it could enhance conversational learning experiences for children. However, the human-like quality of Dessa has also raised concerns about the technology's potential for misuse, such as facilitating more convincing phone scams.

Critics argue that the ability to generate speech that closely mimics human speech could make it easier for scammers to deceive unsuspecting individuals. Block has not disclosed specific details about Dessa's capabilities, such as its size, benchmark results, or a public release date. The company is currently offering limited access to the AI to a select group of AI safety researchers.