Separate Dialogue, Music, and Sound Effects in ComfyUI
A video arrives with one mixed soundtrack. You want to replace the dialogue, but keep the background music and sound effects. Muting the audio removes all three. I built ComfyUI-DBS to bring that separation into a ComfyUI workflow. It splits a soundtrack into music, sound effects, and speech , then lets you save the parts or combine them again. The repository includes a workflow that writes a new…
A video can contain mixed audio, including dialogue, background music, and sound effects. You might want to separate the dialogue from the other audio components while preserving the music and sound effects.
This is where ComfyUI-DBS comes in handy. This ComfyUI workflow splits a soundtrack into three distinct components: music, sound effects, and speech (dialogue). It then allows you to save these individual tracks or combine them again to create a new video with the modified soundtrack.
ComfyUI-DBS is based on the Dynamic Band Splitting (DBS) technique developed by Bingzhe Zhang, Kai Li, and Xiaolin Hu. The integration brings the power of DBS into the ComfyUI workflow. The two available models are relatively small, totaling around 23 MB, and can be run locally on either a CPU or an NVIDIA CUDA GPU.
The DBS technique separates the audio into three components: Music (the musical part of the soundtrack), SFX (the sound effects), and Speech (the spoken dialogue). The workflow then combines the music and sound effects to create a new soundtrack for the video, effectively removing the dialogue. If you need a separate file with just the speech, the workflow can save that as well. Additionally, you can adjust the relative levels of these individual audio tracks if needed.
Before processing a longer video, it is advisable to test the workflow on a short excerpt to ensure satisfactory results. For instance, a 12-second excerpt from the film "Tears of Steel" demonstrates the effectiveness of the DBS-TIGER model when run on a CPU.
Installation of ComfyUI-DBS requires ComfyUI version 0.38.0 or later, with the V3 node API, and Python 3.10 or later. While a CUDA GPU is optional, it can significantly speed up the processing time. To set up ComfyUI-DBS, clone the repository from GitHub, install the necessary Python dependencies, and download the two available models from the provided script.
After installation, you can use the supplied workflow file to separate the dialogue from your video, which will save the music and sound effects as a new soundtrack while also storing the dialogue as a separate FLAC file.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
