Building FrameMill: Turning Hours of Gameplay Into an Editable First Cut
Written for the Gemini All Agentic AI Hackathon. A game stream can last four hours. The moments worth turning into a video might last only a few minutes. Finding those moments usually means rewatching the recording, marking timestamps, cutting clips, and rebuilding the story manually. Existing AI clipping tools can reduce that work, but they often return disconnected clips or a finished video…
FrameMill is an AI-powered tool designed to convert lengthy game streams into editable video projects. The process begins with the detection of gameplay activities and timestamps using a deterministic Go processor. This information is then passed to Google ADK agents, which employ Gemini to map the game and plan an edit. The Editor agent constructs a validated Timeline with provenance, which is then rendered as a preview by FrameMill.
The creator can modify the Timeline and initiate a new render if needed. The backend is built with Go, utilizing the local server, processing core, and agent runtime, while the frontend is constructed using Vue, TypeScript, Vite, Tailwind CSS, and Pinia. FrameMill operates on the MediaMTX, Cloud Storage, Eventarc, Cloud Run, and Firestore architecture.
It processes footage by performing sparse checks and refining denser segments of interest, utilizing persistent decoder contexts to avoid repeatedly opening the same media. The Timeline is the canonical result, not the generated video, ensuring a high degree of editability and trust in the AI-driven editing process.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
