Bug0 Releases QA Agent to Automate Regression Testing With Cached Playwright Replays

Bug0 has launched a QA Agent, a Playwright library that automates regression testing by caching Playwright replays with AI. The agent takes natural language test instructions, associates them with UI actions, and caches appropriate steps for faster CI runs. It uses consensus from up to three LLMs and vision model evaluations for verification.

Legacy testing approaches, while automated, lack AI integration and often struggle with implementation detail changes and flaky tests. In contrast, fully agent-based automated testing dynamically handles browser interactions. However, AI-first testing suffers from high latency, making it impractical for large regression suites. Bug0's hybrid approach separates visual exploration from routine regression execution.

Developers write declarative test steps in plain English inside Playwright specs. During initial execution, the agent inspects the accessibility tree and screenshots, performs requested actions, and caches successful single-action steps in an in-memory Redis store. On subsequent runs, cached actions are replayed through Playwright, avoiding LLM inference latency.

When UI changes occur, the replayed Playwright action is detected as a failure, and the cache is invalidated before re-invoking the multimodal agent loop. Assertions, however, are not cached and are executed through vision models on each pass. Claude and Gemini independently evaluate assertions, with a tie-breaking third model. Video models help detect transient UI defects.

The QA Agent is released under the Functional Source License, Version 1.1, ALv2 Future License, which imposes non-compete restrictions for the first two years of usage.