Turning a video clip into a game sprite sheet in the browser
I make 2D characters by generating a short animation video and cutting it into a sprite sheet. For a while that meant ffmpeg, a Python script and a lot of fiddling with transparency. I moved the whole thing into a browser page so it runs on the user's machine and the video never leaves it. This post is what I learned along the way: reading frames from a <video> , removing a flat backdrop from…
1. Reading frames: To extract individual frames from a video clip, one needs to manipulate the video element's `currentTime` property and wait for the `seeked` event to fire. Once the desired frame is reached, the video is drawn onto a canvas element, and the pixel data is retrieved using `getImageData`. This process is repeated for each desired frame, with the resulting frame data stored in an array.
2. Background removal: Since the background is typically flat or transparent in 2D character animations, the script looks at the four corners of each frame to identify the backdrop color. For PNG sheets, the corners must match within a tolerance of 12 per color channel. In the case of compressed video frames, a larger tolerance of 24 is allowed, and the average of the corner colors is used as the backdrop reference.
3. Code implementation: The entire process is implemented using plain JavaScript with no external dependencies. The `readFrames` function takes the video element, an array of frame times, and the desired output width and height as parameters. It creates a new canvas, sets its dimensions, and initializes a 2D rendering context. The function then iterates through the specified frame times, seeking to each time and drawing the video frame onto the canvas before retrieving the pixel data. The function returns an array of frame data, which can be used to create a sprite sheet.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
