Sub‑50 ms On‑Device TTS: Instant Voice for Games & Streams

Instant voice generation on devices with sub‑50 ms latency is now a practical reality thanks to advancements from major tech firms and open‑source communities. Ultra‑low‑latency text‑to‑speech (TTS) enables NPCs to instantly respond in games, live‑streamers to overlay multilingual audio without noticeable delay, and streamers to dub or subtitle content in real time—all within a few frames of video.

Recent releases from Nari Labs, Meta, and open‑source projects have made high‑quality, ultra‑fast models accessible. To achieve sub‑50 ms performance, developers should focus on optimizing model size, using hardware acceleration, and quantizing the model for speed. The fastest open‑source models currently available include VITS‑Lite (38 ms on Apple M2), FastSpeech‑2+ (24 ms on RTX 4060), and Glow‑TTS‑Tiny (31 ms on Intel i7‑12700).

These models operate within the CPU/GPU compute stage, excluding network latency, by using techniques like int8 quantization and batch size of 1.

Deploying an ultra‑low‑latency TTS service involves several steps. First, install Python 3.10+ and necessary libraries such as ONNX Runtime GPU, NumPy, SoundFile, and tqdm. Download a quantized model like VITS‑Lite, which is available in ONNX format and compressed to just 45 MB. Quantize the model to int8 using ONNX Runtime tools for further speed gains.

The inference script loads the model, processes input text into tokens, measures the time taken for inference, and streams the resulting audio using PyAudio for real‑time playback.

For example, on an Apple M2 laptop, the script can output an inference latency of approximately 37.8 ms, confirming that sub‑50 ms TTS can be achieved on consumer hardware. This capability is crucial for gaming, live streaming, and other applications where real-time audio is essential, offering significant privacy benefits by keeping all processing on‑device and potential cost savings by eliminating cloud inference.