How I Built an Interactive 3D Short Film You Can Talk To

"I’ll Call You Later" is an interactive 3D short film that can be watched directly in a web browser with no installation required. The film runs for about four minutes and ends with an option for the viewer to engage in a live voice conversation with the character. The film was created by Jas (Jashwanth Palleti), who wrote, directed, performed in, and scored the entire production, including the English and Telugu music.

Jas utilized an AI coding assistant called Claude Code to assist with the majority of the coding process. The story revolves around a man driving home from work around 9 PM, caught in traffic in a US tech city. His mother calls and he says, "I’ll call you later." He then makes a wish under a set of aligned stars, effectively ending the first part of the film with a "to be continued" message.

The creation of the character in the film involved a complex process that started with phone videos. These videos were processed using YOLOv8 person detection and an IoU tracker, resulting in 133 whole-body keypoints extracted using ONNX Runtime on a GPU. These keypoints were then fed into a SMPLify-X-style SMPL-X fitter in PyTorch, which in turn uses a VPoser pose prior, a robust 2D loss, and temporal smoothness.

Calibration was done by recording a calibration take to fix the body shape. The process was split into chunks to fit within the memory limit of an 8 GB GPU. The fitted motion was then ground-truthed to estimate the floor plane from the planted feet and used heel and toe contacts. The retargeted motion was finally applied onto the character rig in Blender, achieving a mean reprojection error of 12.9 px.

Grounding reduced foot slide from 4.96 to 2.09 mm/frame. This complex process, however, also had its obstacles. For instance, the avatar once read as tired when the creator was actually surprised, even though the metrics appeared fine. A particular motion, getting out of the car, was too challenging and required using a Mixamo clip instead of the fitted take.