No Mic Needed: You Can Create Music and Speech With Adobe’s AI Audio Tools

Adobe has unveiled new AI audio tools that enable users to create speech, music, and sound effects without needing a microphone. Adobe's head of AI audio, Jay LeBoeuf, explained that these Firefly AI tools provide "a very stable foundation" of audio options for creators, catering to filmmakers, musicians, and other professionals. Unlike other AI music generators, Adobe's offerings focus on more targeted, professional-grade tools for various applications.

One of the primary uses of these AI audio tools is to transform creators' scripts into audio for overlaying on TikTok videos or generating custom soundtracks without worrying about copyright infringement. The tools are integrated within Adobe's AI hub, Firefly, which may already be available in a user's Creative Cloud subscription or can be accessed with a Firefly-only subscription starting at $10 per month.

To generate speech, users can upload a script and choose from various artificial voices in different genders and ages, with the option to translate the audio into over 20 languages. Pronunciation guidance is also available for specific names or products. Adobe's AI audio model is designed to recognize and apply emotions to its outputs, making voices sound more natural and less robotic.

Adobe's AI audio policy states that it only uses licensed and publicly available content and data for training its AI models, and it never trains on customers' work. When creating music or soundtracks, the tools generate instrumental background audio, not full AI songs, which are primarily used as background music for videos. Users can input a video and let the AI create four sample audio tracks that fit the video's vibe, ranging from 10 to 30 seconds long.

Sound effects are the final component of the AI audio triad and can be created by uploading a video or recording a human voice, which the AI will transform to match the desired sound effect. All audio generated by Adobe's AI is commercially safe, and the music is automatically granted a universal license, eliminating concerns about copyright infringement and potential lawsuits.

Adobe's new AI audio tools were first teased at last year's Adobe Max conference and are now generally available, further integrating generative AI into Adobe's flagship programs. A recent survey from Berklee College of Music revealed that nearly one in five musicians and creators use generative AI at some point in their process, with one-third using AI-generated audio in their final cuts.

However, AI audio, particularly music, remains a contentious area, with issues surrounding its authenticity and the need for clear labeling to distinguish AI-generated music from human-created music on streaming platforms.