Google Trends API: the 200 OK that means you got soft-blocked
Google Trends has no public API. What it has is the same internal JSON endpoints the trends.google.com single-page app calls — and those endpoints do something most REST clients aren't built to survive: they answer with HTTP 200 and an empty body when Google decides you look like a bot. Quick answer A 200 OK from Google Trends' widgetdata endpoints does not mean you got data. If the response body…
Google Trends lacks a public API, but it does provide internal JSON endpoints that some REST clients struggle with. When Google suspects a request is from a bot, it returns a 200 HTTP status code with an empty body instead of a 429 error. To handle this, instead of solely relying on the status code, check if the response body is empty. If it is, rotate the proxy session and retry, as this is functionally equivalent to a 429 error.
The JSON responses include an XSSI-protection prefix to prevent cross-site script inclusion attacks. However, this prefix isn't consistent, so parsing the JSON body might fail due to the incorrect prefix. To handle this, strip the XSSI prefix before attempting to parse the JSON.
An issue arose when Google Trends renamed an endpoint without proper announcement. The endpoint 'multiGeo' was renamed to 'comparedgeo,' and there was no deprecation notice. When the old endpoint stopped working, the scraper adapted by updating the endpoint map with a comment detailing the change.
The related queries and topics sometimes return the string 'Breakout' instead of a numeric value when search volume increases drastically. To handle this, treat 'Breakout' as a valid value within the 'int | str | None' type.
To stay under Google's 5-keyword comparison limit, split keywords into batches, with a 2-second pause between batches. This prevents the requests from appearing as a burst and ensures smooth operation.
Scraping Google Trends is legal, as the data is freely accessible through the website. The scraping is done by rotating browser fingerprints and residential proxy exits, back off exponentially when encountering soft blocks, and following standard care practices.
Lastly, if the 'Interest by region' widget returns no data for a keyword, check the run logs for an explicit error rather than an empty result. Values in the dataset represent relative interest scores (0-100) normalized for the specific keyword set and time window. Scores cannot be compared across separate runs, as they are relative to the specific request. Each row in the dataset represents interest for a single keyword within the specified time window.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
