Adobe Firefly Can Now Generate Music, Voiceovers and Sound Effects

Adobe has launched its Firefly audio generation tools for general use, adding AI-created music, speech and sound effects to its suite of image, video and design tools. These new tools operate through distinct AI models and can be accessed within Firefly, eliminating the need for creators to use separate applications or services for audio content creation. The primary objective behind these additions is to streamline the content production process, allowing creators to work within a single platform.

Firefly’s Music Model employs Adobe’s proprietary technology to produce original tracks based on user prompts. Moreover, this tool can tailor music to the length and ambiance of an uploaded video. The Firefly Speech Model, powered by the Firefly Speech Model, allows users to customize voice, pacing, and emotional tone. An alternative model, ElevenLabs, is also provided by Adobe.

The Firefly Audio Model, meanwhile, produces sound effects that align with the actions and timing observed in video clips. Adobe asserts that these tools cater to a variety of multimedia content types, including social media videos, vlogs, short films, product tutorials, and podcast excerpts.

Perhaps the most significant advantage of Adobe’s Firefly audio generation tools is the licensing they offer. Adobe positions music created using its Firefly Music Model as universally licensed and permissible for commercial usage. This is achieved through training the models on licensed material and public domain content, rather than indiscriminately scraping data from the web – a practice that has led to copyright disputes in the broader AI music industry.

However, Adobe’s commercial safety assurances pertain only to its own Firefly models. Users must independently verify the appropriateness of outputs from partner models, such as those from Google, ElevenLabs, OpenAI, Runway, and Luma AI.

Adobe has further augmented its Firefly suite by introducing a complimentary version of Firefly AI Assistant. This tool offers daily generation limits without a paid subscription and can generate storyboards, brand kits, branded mockups, and conduct batch image edits. Additionally, Adobe integrated Google’s Gemini Omni Flash, which accommodates text, image, video, and audio inputs.

The company justified these enhancements by citing a Berklee College of Music study involving 1,003 creators and industry professionals, underscoring the growing demand for music in video production. While Adobe funded the research, Berklee College of Music emphasized that Adobe had no influence over participant recruitment, responses, or analysis.