Automatic lip sync animation generator for Blender animations
I don't know if anyone else here has ever used Blender or tried their hand at 3D animation, but if you have, you know it's hard. Like really hard. I've been using Blender for well over 5 years, and I've definitely improved at animation, but there's one thing I've never gotten right: facial animation. Making and animating detailed facial rigs or shape keys to line up your character's mouth to a…
Blender, a popular 3D animation software, can be quite challenging to master even after years of practice. One aspect that often proves difficult is facial animation, specifically aligning a character's mouth to a voice recording with intricate facial rigs or shape keys. Although there are add-ons and other programs like Adobe Character Animator that provide similar functionality, none seemed ideal for those who want a straightforward solution without having to deal with rigging or mesh deformation, or paying for Adobe licenses.
The author set out to create a tool that could simplify facial animation in Blender, and they succeeded. Their solution involves using Rhubarb to analyze audio input, allowing users to choose pre-made mouth poses or upload their own. The app then generates an image sequence that can be easily incorporated into any 3D program of choice by adding it as an image texture to the character.
The tool is lightweight and user-friendly, making it perfect for creating Lego brickfilm animations in Blender, but it could also work well for other 3D, cartoony, or even 2D animations.
The app is built using a Supabase backend with OAuth authentication, written in TypeScript and CSS, and utilizes a Rhubarb WASM package. It is hosted on Vercel, ensuring fast and reliable performance. Interested users can access the free tool by uploading an audio file (up to one minute long) and their drawn mouth shapes or using the default poses, after which they can download the resulting image sequence.
The author has successfully demonstrated the tool's capabilities in a one-minute YouTube demo, and it is available for free use at the provided link. This innovative solution aims to make facial animation more accessible and efficient for artists working with Blender and other compatible 3D animation software.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
