I Have No Coding Skills. My Team Is 3 AI Models. Here’s the 100-Bug Audit That Saved My Second Startup.

Two weeks ago, I shared my experience building ThumbRank with a trio of AI models and no coding expertise. The feedback was overwhelmingly positive, not because AI could write flawless code, but because AI produced code that appeared flawless until revenue came into play. I didn't stop there. I set the same team to tackle a more significant challenge: GrantPro AI, a funding intelligence platform.

Our core principle was straightforward: real numbers or guesses—no middle ground. We used deterministic code to calculate the actual chances of winning grants, relying solely on millions of public award records from sources like USAspending, EU CORDIS, NIH, and UKRI. We maintained the same team architecture: AI #1 for feature development (React, Supabase, Edge Functions), AI #2 for design and infrastructure, and AI #3 for auditing every line of code shipped by the first two.

Initially, things appeared to be moving smoothly. We had a data pipeline, a dashboard, and a Paddle checkout. However, when I handed AI #3 the codebase before any potential customers could purchase, it returned approximately 100 unique bugs and 56 issues concerning code quality. Some of these bugs could have crippled the product from day one.

Here are the most critical ones, the issues that could have jeopardized the project, and how we resolved them.

The first issue was the $0.00 Paddle Webhook Disaster. The AI configured the Paddle webhook to listen for price IDs but mistakenly mapped two products to the same priceId. This resulted in users purchasing the 1-credit pack receiving 5 credits, while those buying the 5-credit pack received none. The frontend displayed "Success," and the webhook returned a 200 OK status, but the database silently discarded the transaction.

The solution was to move the idempotency check to the database level, using a unique index on the credit transactions table. This way, if a webhook event arrived twice or if there was a mapping error, the database would throw an error and log it.

Another significant problem was the Client-Side Credit Spending Glitch. The AI wrote the credit deduction logic in the React frontend. When a user with no credits clicked "Generate," the UI allowed the action, and the Edge Function was called. This vulnerability meant anyone could bypass the UI by opening the browser console and directly calling the API, potentially draining our OpenAI API credits.

To fix this, we removed all client-side credit logic. Now, spending happens exclusively through a Postgres SECURITY DEFINER RPC that checks the user's balance atomically on the server. This prevents unauthorized access and ensures that only legitimate transactions proceed.

The Public Data Moat Leak was another critical bug. We spent a month ingesting 500,000 grant records into tables like awards and win_assessments. While we used Row Level Security (RLS) to protect our data, we left the SELECT policy open to the anon role for caching purposes. This oversight allowed anyone with a Supabase anon key (public by design) to access our proprietary data empire.

The fix was to drop the public read policies. Any table containing secret sauce—historical awards, win probabilities—must be locked to service_role or authenticated users only.

Lastly, there was the Zombie Function. The AI constructed an entire compliance checker, complete with a beautiful UI, complex prompt, and Deno Edge Function. However, the AI forgot to export the handler, causing the function to compile but fail silently when called by the frontend. This was a zombie feature, essentially non-functional.

To prevent such oversights, we added a CI gate that explicitly tests the exports of every Edge Function before deployment. By understanding these bugs and their fixes, we learned that while AI can write isolated files efficiently, it struggles with understanding cross-file intersections. Our solution was to build rigorous systems to catch these issues.

We created an "Invariants Doc" outlining rules that must be adhered to across all files. Every paid checkout must result in credits being assigned to the correct identity, and no public table can be writable using the anon key. AI #1 and AI #2 follow this document before coding, and AI #3 uses it as a reference before auditing. We also kept the old, buggy commit of the auditor.

Every time we updated the auditor's prompt, we ran it against the old code to ensure it still caught the 100 bugs. If it didn't, the prompt was too lenient, and we rejected it. By building these systems and adhering to strict rules, we transformed our approach to coding with AI from a potential liability to an integral part of our product's security.