{
  "id": 14881907,
  "title": "Apple's App Store review feed has holes — and whether you hit one depends on your HTTP client",
  "url": "https://urgent.news/2026/10/16/apples-app-store-review-feed-has-holes-and-whether-you-hit-one",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-16T06:01:36.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/fetchsmith/apples-app-store-review-feed-has-holes-and-whether-you-hit-one-depends-on-your-http-client-oc0"
  },
  "original_language": "en",
  "account": "Apple's App Store review feed has inconsistencies and accessibility depends on the HTTP client being used, according to recent findings. The feed, which can be accessed via the URL itunes.apple.com/country/rss/customerreviews/id=.../sortBy=.../page=n/json, appears to have empty pages appearing randomly throughout a result set, rather than at the end. These empty pages can cause scrapers to prematurely end their data collection, reporting \"no reviews found\" when, in fact, reviews exist.\n\nThe issue is not limited to a single device or browser, but rather the HTTP client used to make the request. For example, a curl request made with a plain HTTP client may return empty pages consistently, while a browser-shaped client with browser-like headers can retrieve the complete set of reviews. This discrepancy between curl and browser headers suggests that the client's shape, rather than the Apple device itself, determines whether a specific page is a hole in the feed.\n\nThe original advice to rotate User-Agent strings to handle empty pages and cache the winning variant proved ineffective. Not only did this approach result in up to four times the number of requests on every empty page, it also concealed the actual cause of the problem. When a scraper encounters four fingerprints and still sees no reviews, it may falsely conclude that the feed is truly empty. In reality, the issue lies in the loop prematurely stopping at page one and never checking subsequent pages.\n\nTo accurately access the App Store review feed, it is recommended to test the anti-bot theory before implementing any scraping machinery. Additionally, it is crucial to test the solution on different days, as the feed's state can change over time. Varying the client class, such as using curl versus real-browser headers, can help predict the presence or absence of holes in the feed.",
  "summary": "Correction, 2026-09-10 (second update, same day). This post has now been corrected twice, and the second correction retracts part of the first. The original (2026-09-09) blamed empty feeds on request-header fingerprinting and recommended rotating User-Agents — wrong. The first correction, earlier that day, replaced that with two mechanisms: feed holes (right, and confirmed again below) and two…",
  "key_points": [
    "App Store review feed has inconsistencies with empty pages appearing randomly",
    "HTTP client determines accessibility of review feed, not the device"
  ],
  "editors_take": "Inconsistent data delivery within the App Store review feed forces developers to prioritize client fingerprinting and anti-bot testing to prevent scrapers from prematurely concluding that review sets are empty.",
  "illustration": null,
  "coverage": {
    "outlets": 2,
    "also_reported_by": [
      {
        "outlet": "9to5Mac",
        "title": "Apple Store is down ahead of iPhone Duo preorder rush",
        "url": "https://urgent.news/2026/10/16/apple-store-is-down-ahead-of-iphone-duo-preorder-rush",
        "published": "2026-10-16T07:02:48.000Z"
      }
    ]
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}