Turning a 5-minute talk into 7 images for an LLM: picking one sharp frame per scene
If you send a video to a vision LLM, the usual first step is to sample a frame every few seconds. On a 4:44 conference talk, one frame every 10 seconds gives you 28 images. The talk has 7 slides. You pay for 21 repeats, and some of the 28 catch a slide halfway through a transition or before its last bullet appears. What you actually want is one sharp frame per scene: per shot in a film, per slide…
Turning a 4-minute conference talk into a single image for an LLM involves careful selection of one sharp frame per slide. By scanning the video once in grayscale, the initial step establishes the cost of the entire process. Sharpness, change, and flatness are computed for each sample, yielding three key metrics. An adaptive threshold is employed to identify scene cuts, distinguishing hard jumps from gradual changes caused by camera motion.
Once scenes are identified, the best frame within each scene is chosen based on sharpness relative to local motion. Frames that are blurry or duplicates are discarded, ensuring only the best frames remain. Finally, full-resolution frames are extracted in a single pass using an optimized ffmpeg process. This method results in one high-quality frame per slide, effectively turning a 4-minute talk into a single 7-frame image.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
