Prompt engineering couldn't fix this LLM bug. 20 lines of binary parsing did.
New here (hey 👋) and wanted to share something. We recently made public an ad intel app we'd been using internally in our agency. The app/system runs video ads through Gemini to pull out a transcript, the hook, and a few timed moments. It works fairly well - especially for ideation. The transcription is genuinely accurate. A user opened a 2:22 video and saw a moment marked at 3:37 . Not a…
A new internal ad intelligence application was recently unveiled, which extracts transcripts and key moments from video ads using Gemini. The transcription proved to be accurate, but a bug emerged when the model incorrectly extended the video timestamps past their actual duration. A 2:22 minute video showed a marked moment at 3:37, which was actually 50% past the video's end.
The model's inconsistent behavior persisted despite various prompt hardening attempts, such as explicitly stating the video duration, setting a timestamp limit, and reiterating constraints. The root cause of the problem was that the model could violate constraints, but code enforcement was necessary to prevent it. The solution involved reading the video's duration directly from its MP4 file, which contains the duration information in its header.
A custom JavaScript function was created to extract this duration from the buffer, ensuring that timestamps were validated server-side before being sent to the client. This approach eliminated the impossible timestamps and preserved the accurate transcription. The incident highlighted the importance of distinguishing between falsifiable and unverifiable outputs, and the need for server-side validation to maintain data integrity.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
