スマホの指がエコーキャンセラを壊していた — 音でないもので発話が切れる

The iPhone Safari browser on iPhones stopped working with a voice dialogue avatar when used on mobile devices. The issue stemmed from the browser's echo canceller being unable to handle sudden changes in sound when the user covered the speaker with their finger, causing the browser's echo canceller to misinterpret the residual sound as human speech. This led to the avatar repeatedly starting and stopping its speech at two-second intervals.

Upon examining the logs, a pattern emerged: a speech start was detected 5 milliseconds before the avatar stopped speaking, and this occurred in all five instances. The decisive factor was that no transcription was output, as the sound was not recognized as a word.

The root cause was the browser's automatic gain control (AEC) not being able to cancel out the echo during playback, leading to the residual sound leaking out. This misinterpretation was seen by the voice activity detector (VAD) as human speech, causing the avatar to stop speaking repeatedly.

The author had previously implemented a gate as a countermeasure against self-echo, which drops speech immediately after the sound finishes. However, this gate did not work in this scenario, as it was designed to let through speech during playback to prevent interrupting the speech.

Furthermore, the author investigated how other web services handle this issue. They found that LiveKit, a web service that keeps the microphone open for conversation, waits for the first few hundred milliseconds of speech before making a judgment, which helps to reject 51% of VAD-based interrupts. However, no silver bullet solution exists for the echo itself, and most solutions rely on the browser's standard automatic gain control, which is a common premise on desktop PCs but not on mobile devices.