Block’s new voice model sounds eerily human

Block, led by its CEO Jack Dorsey, has unveiled a new voice model called Dessa, developed by the company's research team. Dessa boasts the ability to sound remarkably human, capturing nuances such as laughter, pauses, and tears. Unlike many competing voice AI models, Dessa operates locally on a single NVIDIA GPU, requiring no reliance on remote servers.

Dessa is a full-duplex model, enabling it to engage in two-way communication seamlessly, similar to OpenAI's GPT-Live model. Block asserts that Dessa's knowledge comes from the conversational data it has been trained on, eliminating the need for additional retrieval systems often found in competing models.

Critics have long complained that current voice AI systems can feel unnatural and annoying due to the text-based pipelines that generate speech. Models like ElevenLabs v4 and Fish Audio Drama 3 utilize voice tags to imbue emotion into their speech, but Block claims that Dessa's emotion is inherently embedded within the model, not reliant on external tags.

Block's researchers have demonstrated Dessa's capabilities through various conversations, showcasing how the bot adapts its voice, cadence, and emotions to suit different contexts.

For instance, a bot in one conversation reveals its fatigue after a long day at work, conveying the emotion through its tone. In another example, a bot demonstrates empathy and emotional breakdowns when comforting a user. Block also showcased a scenario where a team member played a game with the voice assistant, further highlighting the model's versatility.

Dessa is currently not publicly accessible, and Block is actively seeking safety researchers to test the model, particularly to identify potential risks where a voice agent could deceive users. This move is particularly significant for Block, given its focus on finance and the delivery of solutions to small businesses. Square, Block's subsidiary, already employs voice AI for order handling.

However, with Dessa's on-device capabilities, Block can expand its reach to a broader array of business and consumer applications at a lower cost.