ElevenLabs’ new v4 speech model supports more expression control and 90 languages

ElevenLabs has introduced two new speech models, v4 and v4 Turbo, on Monday, with enhanced features for more expressive control and reduced latency for voice agents. Last year, the company launched its v3 model, and previously hinted at v4 at an event in Warsaw. The new v4 generation employs a novel architecture allowing for improved control and faster cloning.

Users can now clone a voice with just 10 seconds of audio using the v4 model. In terms of creative aspects, the model better interprets voice identity over extended text chunks and maintains context while reading aloud to alter expressions. To enable users to define expressions, ElevenLabs has expanded inline tags in v4, permitting the stacking of multiple tags and the model's adherence to the sequence.

Previously supporting 70 languages, ElevenLabs has expanded this number to 90 with the latest version. Quality improvements were observed most significantly in Japanese, Brazilian Portuguese, Mandarin, and Cantonese. The startup has seen rapid growth in its enterprise calling business, with over 55% of its revenue now coming from large companies.

The new model caters to voice agents, thanks to its lower latency, allowing for more fluid conversations. Moreover, v4 can initiate audio generation as soon as the underlying Large Language Model begins generating responses, and it can better handle confrontations, escalations, and different outcomes to improve issue resolution.

With fierce competition among startups like Cartesia, Deepgram, Fish Audio, Boson, and WellSaid Labs, as well as industry giants such as Google and OpenAI, ElevenLabs recently raised $500 million from Sequoia, valuing the company at $11 billion. Rumors of a follow-up fundraising round, potentially valuing the company at $22 billion, have already surfaced.

Additionally, ElevenLabs' annual run rate has surged from approximately $330 million at the beginning of the year to over $600 million, and the company's workforce has expanded across various markets, including India, Europe, and Brazil, reaching over 800 employees. In a recent interview with TechCrunch, ElevenLabs' CEO Mati Staniszewski expressed the company's ambition to pursue an IPO in the coming years, though no specific timeline was provided.