Testing Android’s On-Device Speech Recognition With Hindi-English Code-Switching

A mother describes her feelings in Hindi with English health terms interspersed into her sentences – it's just how they speak. The author built an Android app to let her log these feelings by voice instead of typing. Since her village connection is unreliable, the app must work offline and the audio shouldn't be sent to a server.

This requires on-device speech recognition. The API exists, but there are many potential points of failure between the API being available and the user being understood. The author's setup used SpeechRecognizer.createOnDeviceSpeechRecognizer() (API 31+) to run recognition locally. Asking for Hindi and enabling offline use worked well, but real testing was needed.

The author tested with her mother speaking Hindi-English sentences and compared the phone's output to her actual words. Some English words were recognized correctly, but "dizzy" was misinterpreted as a letter combination rather than the intended word. Switching to a more precise language detection mode fixed this. However, this increased latency, which was acceptable for a 15-second check-in that doctors later read.

The author only validated Hindi with her mother's actual speaking, as English (en-IN) needs its own real speaker test and wasn't claimed as validated. Pairing English and Hindi profiles was considered, but dropped due to potential misinterpretation of English as Hindi. The author also considered adding a feature to let users reject the transcript, as on-device mode keeps no audio and a wrong transcript would be unrecoverable. The app is built strictly for logging symptoms, never diagnosing.