Urgent.News

What's breaking now, across thousands of outlets.

Tech

The last mile of a scraper is a human copying a spreadsheet

I scrape product cards from online stores for a content bank with an Actor of my own . The Actor finishes, the dataset fills up, and then a person exports it, checks it, and pastes it into the tool where the work actually happens. That last step has no technology in it. It is somebody's Tuesday. When Apify shipped MCP connectors , I wanted to delete that step. A connector lets an Actor call a…

The final leg of a scraper operation is performed manually by a human, who copies a spreadsheet after the AI completes its task. This step lacks any technological assistance and is part of a human's typical Tuesday. Apify's MCP connectors were designed to eliminate this manual process. A connector enables an Actor to communicate with a third-party service using the Model Context Protocol and the user's authorized credentials.

The Actor never stores the token and communicates with an Apify proxy, which injects the real credentials on the server side. To demonstrate the functionality, a small Actor called "card-sink" was created to take product cards and write them into a user-selected service. The code is available on GitHub. Six failed runs led to various issues, including version mismatches and API errors.

The version conflict arose from using outdated packages. The actor's design allows it to adapt to different destinations without prior knowledge. It asks the connected service which tools it offers, selects one that can write, and shapes the call accordingly. The connector is defined using an input field with the resourceType "mcpConnector".

The MCP server list filters and limits the user's connectors and the actions the Actor can perform at runtime. A mistake in the code resulted in an empty dataset when no connector was selected, while the warning was logged instead. This led to an incorrect assumption that the Actor had nothing to do.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

Revisiting Joel's Test - exe.dev blog

  • Joel Spolsky's 12-step test remains relevant despite tech advancements
  • Shelley Test replaces human code review with LLM adversarial reviews
  • Athena agentic bot supervises continuous deployments and logs

Two Actors, one agent, and the three ways my chain broke

A client asked a question that sounds simple: which sellers on the marketplaces are risky to buy from? Answering it takes two steps.

  • Two actors processed different task parts, but agent failed to connect them properly
  • First actor scraped sellers with tax ID, legal name, store URL; second actor calculated risk scores
  • Data format mismatch, empty strings as input caused incorrect risk scores and chain failure

More from Wednesday 2 September →