Fourteen Speaker Encoders Heard the Same Voice. Their Error Rates Differed Five-Fold.
A speaker encoder turns a few seconds of speech into a vector, and the distance between two vectors is treated as an answer to "are these the same person." That number gets used for authentication, for voice-cloning pipelines, and increasingly as the yardstick in papers asking whether a synthetic voice preserved someone's identity. The number is not neutral. It is a reading from an instrument,…
Fourteen speaker encoders evaluated the same voice, but their error rates varied significantly, spanning five-fold. This inconsistency arises due to the encoder's contribution to the authentication process, where the distance between two speech vectors indicates whether they belong to the same person. To understand the impact of the encoder, the researcher ran fourteen encoders over 11,935 trials, using a frozen corpus to ensure consistency.
The emotional speech corpus utilized identical sentences across different emotions, enabling the model to learn angry vocabulary and delivery, thereby separating them afterward. Eight speakers recorded 1,360 sentences in six phonation states—neutral, happy, angry, scared, shouting, and whisper—while maintaining fixed lexical content.
The enrolment speech was neutral, while the test assessed the encoder's ability to recognize the person while they were shouting or whispering. The trial list remained constant, allowing for paired comparisons on every encoder. The headline reveals that the error rates ranged from 0.047 to 0.233 across the panel, demonstrating the five-fold spread.
This discrepancy is larger than most architectural differences people argue about. CAMPPlus, the encoder with the highest error rate, consistently demonstrated the 13th highest error among the fourteen encoders, while the lowest-error encoder showed the smallest error. The mechanism behind this disparity lies in the encoder's ability to compress the embedding space, where different speakers cluster together, as evidenced by their proximity in cosine space.
This phenomenon is apparent even in neutral speech, with EER values as low as 0.031, where several ReDimNet checkpoints reach 0.000. However, expressive speech exacerbates this issue. Despite its widespread use in open-source TTS systems, the specific effect on audio quality remains unmeasured, as the current study evaluated the encoders in isolation on human recordings.
The parameter count does not correlate with the expressive error rate, with the 20.8M model ranking 13th and the 4.81M model leading. The claimed spectral effect, which attributed an 11.7 dB difference to spectral denoising, was retracted after investigation, revealing that the effect was not significant. The researcher stresses the importance of validating the instrument before measuring with it, as the instrument's performance greatly affects the error rates.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
