Read freely, confirm before writing: a safety model for computer-use agents

Every few weeks, the author hits a roadblock when trying to pull data from tools that lack a usable API. This issue is common in seller centers, supplier portals, and 3PL dashboards. To address this, they created Webhands - a computer-use agent that navigates these dashboards using a real headless browser, retrieves clean structured data, and includes a safety feature that only allows write actions with explicit confirmation.

The safety model relies on "recipes" which consist of an entry URL, optional login and navigation steps, and an extraction specification. A crucial part of this model is the write?: boolean flag on the click action type. This flag marks a step as a write, which is the only action that can change the state of the system. Without the confirm: true flag, Webhands will refuse to execute the write step.

This design choice ensures that reads remain safe by default, while writes require deliberate confirmation, minimizing the risk of unintended mutations. On a successful run, Webhands provides the extracted data as structured JSON, a screenshot of the browser's view as a base64 PNG, and a log of the actions taken during the run. These features help verify the results and provide transparency.

The agent runs on Cloudflare Workers with Browser Rendering, offering two modes: live mode, where it executes the recipe using a real headless browser, and dry mode, where it simulates the run without actually executing any actions. This allows developers to test and build the recipes without incurring costs, and then switch to the live mode when ready to execute the changes.

However, it's essential to note that the safety model relies on the recipe author's explicit labeling of write steps. If a destructive button is marked as a read or not marked at all, Webhands will execute it without confirmation, as the agent treats any click as navigation unless explicitly flagged as a write action.