{
  "id": 21036857,
  "title": "Vibe coding testing: check your app before users do",
  "url": "https://urgent.news/2026/11/21/vibe-coding-testing-check-your-app-before-users-do",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-21T11:19:41.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/pantheontable/vibe-coding-testing-check-your-app-before-users-do-2i4m"
  },
  "original_language": "en",
  "account": "Andrej Karpathy, a computing expert, built a website using an AI tool and felt it was 80% complete. However, he later realized it was closer to 20% done. He advises that clicking around only verifies paths you have tried, so AI testing involves giving the AI a way to run tests and using the app as a customer would. It is crucial to ask the AI to write tests for core actions, show the command and output, and have a fresh review session. Test every change on a preview copy before deployment and collect user errors to know when something breaks. Clicking through the app is insufficient; you must test for empty forms, double clicks, weak connections, and second accounts. Karpathy warns that clicking through gives a false sense of completion, and AI tools are good at simple tasks but struggle with complex system integrity. To check AI-written code, provide a way to know if it succeeded, like a test file, build, linter, or screenshot. The AI should explain the code and run the tests, providing exact commands and outputs. Non-developers should seek expert opinions to validate AI output.",
  "summary": "Originally published at pantheontable.com/blog/vibe-coding-testing-check-your-app-before-users-do Andrej Karpathy vibe coded a website and felt 80% done. By his own account, it was a bit closer to 20%. If you built an app with an AI tool and it seems to work when you click through it, this is for you. Below is what Karpathy, David Heinemeier Hansson, John Carmack, Joel Spolsky, Simon Willison,…",
  "key_points": [
    "Andrej Karpathy built a website using AI and felt 80% complete.",
    "AI testing requires giving AI a way to run tests and use the app as a customer.",
    "Clicking through app is insufficient; must test for empty forms, double clicks, weak connections."
  ],
  "editors_take": "Relying on manual clicks creates a false sense of progress that masks deep systemic flaws, forcing developers to adopt rigorous, automated testing protocols to validate AI-generated software.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}