Urgent.News

What's breaking now, across thousands of outlets.

Tech

Ten Lines That Fix Proxy Rotation in a Web-Scraping Platform (issue assignment PR, in 2 hours)

Last night I browsed open issues in a crawler platform (Maxun) and found one proposing a {session} placeholder in proxy usernames — the trick residential-proxy providers use to give each run its own sticky exit IP. Nobody had answered it in two hours. I read the code before replying, which is the whole trick to being useful on the internet: The proposal's call-site map was real:…

A Maxun crawler platform issue proposed inserting a placeholder in proxy usernames to give each run its own sticky exit IP. After reading the code, the reporter replied with a solution that fixes proxy rotation in the web-scraping platform. The fix involves generating a random session ID, replacing the placeholder in usernames, and saving the session ID per run. The maintainer approved the PR two hours after the comment, and the patch only affects users who opt-in to the feature, ensuring it's a no-op for others.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

10 Curated PMP Mistakes That Drop Paid Filters and Kill Fill

On October 6, 2026 , the IAB Tech Lab opened a public comment period aimed at simplifying programmatic plumbing. Curated private marketplace lines are a big part of that conversation: buyers pay for…

  • Curated private marketplace lines play significant role in programmatic plumbing streamlining.
  • Upstream filter becomes non-existent when operations treat structure as generic PMP row.
  • Leaving privateauction unset allows open auction competition on invite-only packages.

robots.txt Is a Contract: Parsing It Without Lying to Yourself

You fetched /robots.txt , got a 200, grepped for Disallow , found nothing scary, and started crawling. That is not reading the contract — that is skimming the cover page.

  • Robots.txt is a contract requiring careful parsing to avoid self-deception.
  • Crawl-delay, user-agent scoping, and contract focus on front door are often overlooked.

The Red Pencil Workshop

The workshop opens on a staged failure, because a visible row count teaches faster than a definition. On the projector, a fake ledger shows three captures for one invoice after a short reply suggested…

  • Red Pencil Workshop uses staged failure to teach concepts
  • Workshop employs red pencil as script exit line
  • Focus on stopping at one process to deepen understanding

More from Tuesday 13 October →