Amazon Deploying AI Tech That Forces Actors’ Mouths to Match Dubbed Dialogue

Amazon is pushing the boundaries of sub versus dub debates with the launch of an artificial intelligence (AI) powered feature that synchronizes actors' mouth movements with dubbed audio tracks. The company's Prime Video streaming service announced this new tool on Thursday, stating that it will exclusively be available on the English dub of the German series "Maxton Hall" initially, but plans to expand to other titles in the future.

Raf Soltanovich, vice president of technology at Prime Video and Amazon MGM Studios, stated in a statement that the company is constantly innovating with new tools to enhance the viewing experience for its customers. The AI technology aims to create a more seamless and immersive way for customers to enjoy global content.

However, there are several questions surrounding the deployment of this feature. It remains unclear how much of the feature is automated by AI, and concerns have been raised about the potential rightsholders' involvement in its implementation. Critics argue that this AI-driven approach could lead to a homogenization of actors' unique facial movements, potentially diminishing the authenticity of their performances.

Furthermore, the company's use of this technology raises concerns about the level of control Amazon has over the presentation of artistic works. The effectiveness of this AI technology is also in question, as Amazon has not provided any clips demonstrating its capabilities.

Amazon Prime Video previously experimented with AI-aided dubbing for a dozen movies and shows in English and Spanish last year. The company has also explored AI-generated show recaps and recommendations based on character arcs and plot points. Despite these efforts, there are valid concerns about the potential erasure of language and cultural differences when it comes to subtitles and dubs. Some argue that treating the language barrier as an issue to be erased and covered up is concerning.