Urgent.News

What's breaking now, across thousands of outlets.

AI

Building a self-healing scraper agent that won't ship a fix it can't prove

I built patchwright for Google's All Things Agentic Hackathon, and this post is my write-up of it, written for the purpose of entering. The short version: it's an agent that repairs broken web scrapers. When a site changes its layout and a scraper stops working, the agent figures out what changed, rewrites the scraper, proves the new version in a sandbox, has a second model review it, and then…

The author developed an AI-powered tool called Patchwright for Google's All Things Agentic Hackathon, designed to automatically repair broken web scrapers. When the layout of a website changes, the scraper often breaks, making it challenging to extract the correct data. Patchwright's goal is to identify the changes, rewrite the scraper code, verify the fix in a sandbox environment, have a second AI model review the fix, and then wait for a human to approve the changes.

The author found that the most critical aspect was ensuring trust in the AI's fixes, as the tool's output must match the original data exactly. They built a simple fake listings site called Fauxpost to test the tool, where they could ensure that the patched scraper's output matched the original twelve records. The project relies on the Gemini 3.7 and Gemma 4 language models, running on Google's Agent Development Kit for JavaScript, and is deployed on Cloud Run with Firestore for state management.

Throughout the development process, the author faced several unexpected challenges, such as the AI's behavior when writing new scraper code, issues with sandboxing, and unexpected interactions between different components. The reviewer component, which generates a short summary of the patch for human approval, turned out to be the most valuable part of the system, as it provides a crucial second opinion before the final approval is given.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in AI

Tooling every AI software harness should have

When an agent writes the code, you stop reading every line. You can pretend otherwise for a while, but on any real codebase you end up skimming a 600 line diff and approving it because nothing jumped…

  • Every AI software harness must include comprehensive tools and checks.
  • Tools should be accessible via a single command and fail to discourage ignoring.
  • Mutation testing, type checking, and linters are crucial for code quality.

More from Saturday 22 August →