{
  "id": 6812317,
  "title": "From 9 Seconds of Voice AI Latency to 1.5 Seconds: Building an In-House Voice AI System",
  "url": "https://urgent.news/2026/09/11/from-9-seconds-of-voice-ai-latency-to-1-5-seconds-building-an-in",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-11T22:22:26.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/mehar_aziz/from-9-seconds-of-voice-ai-latency-to-15-seconds-building-an-in-house-voice-ai-system-bal"
  },
  "original_language": "en",
  "account": "Nine seconds of silence was the wait time callers experienced after asking the AI assistant a simple question, often causing them to hang up. The project began as an attempt to integrate VAPI, a Voice AI platform, into the existing system for automating user onboarding. However, the client's request for a more natural voice led to the realization that customizations would be required beyond what VAPI could provide. The team explored Twilio, Deepgram, and Cartesia as alternatives, ultimately deciding to build their own Voice AI system to maintain control over the orchestration layer. Optimizing for time-to-first-audio instead of total response time was crucial, and they achieved this by implementing conditional RAG, sentence-level streaming to text-to-speech, warm connections, and on-device embeddings.",
  "summary": "Nine seconds of silence. That's how long a caller waited after asking our AI assistant a simple question like \"What's the TB test process?\" long enough that most people would hang up, assuming the call had dropped. We got that down to about 1.5 seconds. This is the story of how a straightforward platform-integration task turned into designing an in-house Voice AI architecture from scratch and the…",
  "key_points": [
    "Callers previously waited nine seconds for AI response, leading to hangups",
    "Team built custom Voice AI system to control orchestration layer",
    "Optimized for time-to-first-audio, achieving 1.5-second latency"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}