Nuance Labs bags $50M in funding to fix the awkwardness of AI avatars

Nuance Labs, a Seattle-based startup focused on improving AI models for face-to-face conversations, has secured $50 million in early-stage funding to achieve this goal. The Series A round was led by Lightspeed Venture Partners, who previously invested in the company's seed round. Existing investors Accel and South Park Commons also participated, alongside Nvidia Corp. and Define Ventures.

The startup aims to create a "human foundation model" that powers avatars capable of engaging in natural conversations with humans, overcoming the lag and awkwardness often associated with existing chatbots. Nuance's approach involves building a more emotionally intelligent avatar by integrating emotional intelligence into the AI model, co-founder and CEO Fangchang Ma explained.

Traditional AI avatars are built through a series of handoffs between different models, leading to delays and a lack of engaging interaction. Ma noted that these handoffs strip away the human-like feel from avatars, making them appear frozen when conversing.

To address this issue, Nuance Labs has developed a full-duplex system that integrates audiovisual perception of the user's stream, allowing the model to react to verbal and non-verbal cues in real-time. This system can perceive words, gaze, gestures, tone, and timing, and respond with facial and vocal expressions. The model learns from studying human behavior during conversations, enabling it to demonstrate understanding in real-time, similar to how humans interact.

The company has released a demo of its model, which shows promising results, with Ma stating that these models enable AI avatars to engage in more natural and productive conversations with humans. Potential use cases include sales, customer service, coaching, professional training, and education. However, Nuance has not yet launched any products as they continue to refine their model to achieve even more fluid conversations.

The funds from the recent funding round will help accelerate development efforts and enable the hiring of additional researchers.