I scanned 13 popular AI-app starter repos. 4 were clean, and the same bug was in 6 of them.
I built a small deterministic scanner for the failure patterns that show up in AI-built apps — the ones that don't stop the demo, but break you in production. Then I pointed it at a corpus of popular open-source starter templates and AI-app repos, because those are what a lot of "vibe-coded" projects are actually built on. If the failure is in the default the framework hands you, it should show…
A small scanner was developed to identify failure patterns in AI-built apps, focusing on default configurations that often persist in well-maintained repositories. This scanner, named vibecheck, was tested on 13 popular AI-app starter repositories and found that 4 out of 13 were free of the identified issues (31%). The remaining 9 repositories contained a total of 101 findings, with 50 critical, 18 high, and 33 medium issues.
Notably, 6 of these repositories had a route lacking authentication checks, 5 contained client-exposed secrets, 5 encountered dangerous HTML rendering, 1 was missing an authentication check, 1 had a wildcard access setting, and 1 included a hardcoded secret. The findings were anonymized, with a summary provided for each project, but the raw data remained available for verification.
The study highlights that these issues are not exotic bugs but rather common defaults that are often overlooked or addressed later. The scanner, vibecheck, is open-source, free, and written in standard Python libraries, requiring only a single second to run.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
