Dropping Candidates Due to Fixable Defects — A Story of Measuring Factory Flaws as Product Personality
📝 Originally published (in Japanese) at forge.workstyle.tech . I Built a System to Mechanically Select Voice Model Candidates I created a system to mechanically select from voice model candidates. It reads probe sentences with 24 candidate voices, automatically measures and scores them based on Whisper match rate, vowel elongation, speech rate, intonation, and jitter, then selects the top…
A voice model selection system created by the author measured candidate voices based on Whisper match rate, vowel elongation, speech rate, intonation, and jitter. The system adjusted weights for different voice roles, such as narrators preferring slower speech and MCs favoring wider intonation ranges. However, the metric proved flawed as it penalized voices that failed to read the script accurately, leading to the elimination of viable candidates.
The author discovered this issue when presenters achieved significantly lower match rates than other roles, despite having acceptable intonation and speech rates. Further examination revealed that hallucinations, not present in the final product, were contaminating speech rate measurements. This defect arose during the voice generation process using Diffusion TTS, where the generated audio was used as training material.
The author found that hallucinations only occurred in the corpus creation stage and not in the trained model used for actual playback. After correcting the probe sentences, the ranking of candidates changed, highlighting the metric's inability to accurately determine voice quality. The author concluded that the system had been training with flawed data, leading to the rejection of candidates with inherent defects.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
