Godot 4 animation retargeting with free mocap: five traps that look like bad animation

Retargeting animation from free motion capture data into a humanoid character within Godot 4 proved challenging due to various pitfalls that resulted in poor animation quality. The five most impactful issues, each accompanied by their causes and resolutions, are detailed below.

1. Wide Legs: The source motion capture file's rest pose did not accurately represent the character's stance in the actual clip. The CMU skeleton's ankles were initially 0.818 meters apart in rest, while the actor's ankles were only 0.268 meters apart. This discrepancy led to distorted leg transfers, with the target character's legs appearing too wide.

The fix was applied to the source import, specifically the SkeletonProfileHumanoid settings (retarget/rest_fixer/fix_silhouette/enable = true), which corrected the source rest pose without altering the clip's world pose. However, enabling this option on the target also moved the leg rest position 32 cm away from its bind pose, causing reoccurrence of the issue.

2. Inverted Arms: The rig used for the character was oriented along the -Z axis, while the motion capture data had its left side on the +X axis. This discrepancy caused the arms to invert during the retargeting process, resulting in hands being positioned far above the hips during poses where the source had them below the hips. To resolve this issue, the character was rotated to face +Z, thereby aligning the rig's orientation with the motion capture data.

Additionally, verifying the character's facing and side positioning using markers within the Godot engine rather than relying solely on render-based estimates helped prevent similar errors in the future.

3. Measurements from Rest Pose in Headless Runs: Measurements taken from the motion capture data using a script in headless mode (godot --headless -s script.gd) were found to be incorrect, as the Skeleton3D.get_bone_global_pose() function returned the rest pose for every frame, even after force_update_all_bone_transforms(). This led to erroneous conclusions being drawn from the data.

To accurately obtain bone rotations, a custom function was created (func chain ( n : String )) that composed the bone transformations manually, ensuring the right results were obtained. This approach provided accurate measurements when the animation was rendered, as the global composition was found to be stale during headless runs.

4. Excessive Twist in Clips: A twist audit conducted using get_bone_pose_rotation() revealed that many clips exhibited excessive twist, ranging from 160 to 180 degrees on almost every bone. The twist measurement included the bone's rest orientation, leading to inflated values. To correct this issue, the delta rotation (difference from the bone's rest orientation) was calculated instead, giving a more accurate representation of the clip's actual twist.

After applying this adjustment, the twist values became more consistent, ranging from 55 degrees in lively takes to 8 to 14 degrees in calmer poses, which aligned more closely with the visual observation.

5. T-pose with Incorrect Arm Rotation: The mesh and armature were rotated separately, causing the exported GLB file to have one arm appearing straight up during animated poses. This issue stemmed from the mesh's bind matrices being influenced by the armature's rotation, resulting in an incorrect pose. To rectify this problem, the mesh and armature should be rotated together, ensuring that the bind matrices remain consistent.

This can be achieved by applying both transformations simultaneously (body.data.transform(mat) and rig.data.transform(mat)), allowing for proper alignment of the mesh and armature in the exported animation. Additionally, verifying the character's pose using the same markers previously mentioned can help avoid this issue in the future.

In summary, these five traps, which encompass issues related to stance, rig orientation, measurement accuracy, twist values, and arm rotation, caused significant challenges during the Godot 4 animation retargeting process. By addressing each trap with the appropriate solution, a more accurate and visually pleasing animated result can be achieved when using free motion capture data within the Godot 4 engine.