google.com/goto: Google's anti-scraping update
Article URL: https://www.autom.dev/blog/google-search-goto-links Comments URL: https://news.ycombinator.com/item?id=49668386 Points: 296 # Comments: 151
Google is introducing a new anti-scraping measure called "google.com/goto" for its search results. Instead of directly showing the destination URL in the HTML, Google is now redirecting users to the actual page, with the target URL encoded in a custom parameter. This new method makes it harder for scrapers to extract URLs from search results.
When a user clicks a result, Google redirects them to the real page, presenting the final destination URL in a structured format. This change is part of Google's ongoing efforts to combat automated SERP harvesting, particularly from AI crawlers and SEO scrapers. The rollout of google.com/goto is now consistent across searches for logged-out and private sessions.
Autom, a developer API provider, has updated its Google Search endpoints to handle these redirect links, ensuring developers can still access usable destination URLs without making changes to their integrations.
Written by urgent.news from Hacker News Best's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
