Speaker-labeled transcription with WhisperX on SageMaker AI

Spoken audio poses challenges for generic speech-to-text systems, such as contact-center calls, all-hands meetings, podcasts, depositions, and broadcast media. Standard transcription methods often fail to provide accurate timestamps and speaker attribution, making transcripts difficult to search, caption, redact, or analyze at scale.

WhisperX addresses these issues by wrapping OpenAI’s Whisper with batched inference, adding wav2vec2 forced alignment for precise word-level timestamps, and incorporating speaker diarization to label who spoke. These features directly apply to real-world workloads, including contact centers, meeting notes, media and e-learning content, and regulated industries like healthcare, legal, and finance.

The AWS WhisperX Deep Learning Container (DLC) is a GPU-ready image containing Whisper, alignment models, and diarization weights, ready for deployment on Amazon SageMaker AI. The container can be deployed as a real-time or asynchronous endpoint, depending on the audio clip length and required interactivity. The real-time endpoint provides synchronous results within a 60-second response time, while the asynchronous endpoint is better suited for long audio files and high-volume batch processing.

To deploy WhisperX, an AWS account, an Amazon SageMaker AI execution role, a GPU service quota for the endpoint instance type, and the DLC image URI from Amazon Elastic Container Registry (Amazon ECR) are required. Additionally, an S3 bucket with “sagemaker” in its name is needed for asynchronous inference. Once the prerequisites are met, a JupyterLab notebook is available to guide users through the entire deployment process in Amazon SageMaker AI Studio.