{
  "id": 1913276,
  "title": "I built a local-first AI video dubbing pipeline with WhisperX + VoxCPM2",
  "url": "https://urgent.news/2026/08/19/i-built-a-local-first-ai-video-dubbing-pipeline-with-whisperx-voxcpm2",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-19T10:28:11.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/kadirb4rut/i-built-a-local-first-ai-video-dubbing-pipeline-with-whisperx-voxcpm2-1bek"
  },
  "original_language": "en",
  "account": "A reporter has built a local-first AI video dubbing pipeline using WhisperX and VoxCPM2. The challenge lies not just in producing one synthetic sentence, but in preserving the original speech timing, background, voice, and rebuilding a video that still feels authentic. This led to the creation of Video Dubbing Translator, a browser-based pipeline for short-form video dubbing.\n\nThe process involves video input, vocal/background separation, translation, voice cloning, fitting the duration, and reconstruction. The interface is designed to show inspectable stages rather than a single opaque \"dub this\" button. While the translation currently uses Google Translate, the media processing and speech synthesis run locally, making the pipeline partially offline.\n\nVoxCPM2 was chosen over XTTS-v2 due to its reference-voice cloning, 48 kHz speech output, and an official model with an Apache-2.0 license. An end-to-end test on a 15-second English clip showed that the translated Turkish result matched the original timings, and the final audio recovered all intended Turkish sentences when run through Turkish ASR. The pipeline outputs H.264/AAC video with 48 kHz stereo audio, with explicit audio contract to catch format problems early.\n\nWhile the current implementation is limited by Google Translate, CPU synthesis speed, and the lack of production-grade speaker diarization, the project is MIT-licensed and open for contributions. The repository includes browser UI, processing stages, setup notes, and demo links. Feedback is sought on voice similarity, timing behavior, installation on different GPUs, multilingual edge cases, and handling multiple speakers. The goal is to create a reproducible local-first workflow that can be inspected and improved upon, rather than a black-box hosted service.",
  "summary": "I wanted to dub short videos without sending the source media to a hosted dubbing platform. The hard part is not producing one synthetic sentence. It is keeping the original speech timing, preserving the background, translating the transcript, cloning a voice, and rebuilding a video that still feels like the same video. That led me to build Video Dubbing Translator , a local-first, browser-based…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}