MTVA-Bench: Evaluating the Language Model Inside Cascaded Voice Agents
Generally, most voice agents are cascaded systems, i.e., an ASR model transcribes the caller's audio, a language model reads the transcript and decides what to say and which backend tools to call, and a TTS model speaks the reply. Nearly all of the decision making happens in the language model, but existing evaluations measure it either too broadly or too narrowly. End-to-end voice benchmarks…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
