Comic Panel Detection With a Vision LLM: Pages Into Shots for Under Half a Cent
How ScreenWeaver detects comic, webtoon and storyboard panels with Gemini Flash: the box format, the cleanup filter, the cost per page and safe retries.
ScreenWeaver, co-founded by the author, uses Gemini Flash to detect comic panels from pages of comic books, webtoons, and storyboards. The process involves sending a single page image to a multimodal model, asking it to provide a bounding box around every panel in reading order, and then cutting those boxes on the user's own server.
This method costs between $0.0036 and $0.0048 per page, making it cheaper and safer than manually cropping each panel with an image editor. ScreenWeaver uses Google's Gemini API, which returns boxes in a 0 to 1000 scale relative to the original image, and the response is validated as JSON. The authors found that contour detection using open-source tools like Kumiko was not as effective as the vision LLM approach, as contours struggled with handwriting, non-border panels, and bleed images.
They also added a caption per panel, which helped the agent that builds shots later understand what each panel is about. The model output is filtered for accuracy and quality, with boxes clamped to the page, swapped edges fixed, small boxes dropped, and captions trimmed. The model call is the most expensive part of the process, but running the import twice can be avoided by using the same tool for manual box creation.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
