Shading Motion

In this article, the author aims to introduce motion as a new creative medium and explore ways to represent it using shaders, referred to as "how to shade motion." The focus is on using WebGPU compute shaders to extract motion through frame differencing and encode it into masks and velocity maps. Unlike luminance or depth, motion is not readily available, so the goal is to create a pipeline that identifies frame changes and maps motion magnitude to a luminance texture.

The article starts with a simple approach, comparing the current frame to the previous one for a grayscale image representing motion magnitude. This method, called frame differencing, works well for extracting motion in 3D scenes, videos, or webcam streams. However, the output may appear flickering and not smooth, which can negatively impact the quality of the desired effect.

To address this issue, the author suggests using temporal decay to accumulate the previous state and fade it over time. This creates a more continuous motion appearance and leaves a trail from fading accumulated frames. Depending on the input image, adjusting the motion threshold and decay rate can help optimize the motion mask for specific scenes.

Once a decent motion mask is obtained, it can be used as input for subsequent post-processing passes to stylize motion. The author outlines how to build the motion mask texture using a compute shader, processing each pixel independently to sample current and previous luminance values, compute differences, update the decayed trail, and write results to a storage texture. This compute shader organizes the motion extraction workflow for React Three Fiber scenes.