Detecting BPM and musical key in the browser (and encoding MP4 with WebCodecs)
We run AIMusicVideoGen , an AI music video generator for independent artists. The main product needs a server, but along the way we built three small free tools that run entirely in the browser: a BPM and key finder, an audio-to-video converter and a Spotify Canvas maker. No upload, no backend, no account. This post walks through how they work, because most of it is plain Web Audio, one…
The article explains how to build three free, browser-based tools for an AI music video generator: a BPM and key finder, an audio-to-video converter, and a Spotify Canvas maker. The tools utilize plain Web Audio, a one-handed FFT, and WebCodecs, and they don't require any servers, accounts, or uploads.
The BPM analysis begins with the Web Audio API's decodeAudioData method, which converts MP3, WAV, and M4A files into an AudioBuffer of raw float samples. The OfflineAudioContext is then used to downmix and resample the audio for further analysis. The BPM detector employs the web-audio-beat-detector package, which takes a tempo range (70 to 180 BPM) and returns the detected tempo. In case the automated detection fails, a tap-tempo button allows users to manually set the tempo.
Detecting the musical key involves creating a chromagram, which represents the frequency content of a short audio segment in terms of 12 pitch classes (C2 to C7). The algorithm slides a Hann window across the mono downsampled audio, performs an FFT for each frame, and normalizes the results. These frames are then correlated with Krumhansl-Kessler major and minor profiles, and the best match is selected as the detected key.
The process is optimized to avoid freezing the tab, reporting progress periodically, and using a Web Worker for better performance.
To suggest chorus or drop locations for a short music video clip, the algorithm computes the Root Mean Square (RMS) energy in half-second blocks and identifies the loudest stretch within the clip length. The user can adjust this selection if needed. For encoding MP4 files, the article describes using the WebCodecs VideoEncoder API, which efficiently encodes frames into MP4 format.
The encoding process takes into account varying frame rates based on the content (30 fps for moving frames, 2 fps for static covers), and the spectrum visualizer for each frame is generated using the same FFT code as the key detector. The article also provides tips for seamless loops, such as maintaining symmetric motion, using keyframe synchronization, and wrapping up the story.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
