{
  "id": 20509659,
  "title": "My Snapchat scraper returned 0 results for a day and still said \"SUCCEEDED\"",
  "url": "https://urgent.news/2026/11/17/my-snapchat-scraper-returned-0-results-for-a-day-and-still-said",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-17T22:58:35.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/zait_dioses/my-snapchat-scraper-returned-0-results-for-a-day-and-still-said-succeeded-3k35"
  },
  "original_language": "en",
  "account": "On Monday, the user checked their Snapchat Spotlight scraper and everything appeared to be working fine. The status was green, the run finished quickly, and the exit code was 0. However, upon opening the output manually, they discovered that the dataset was empty. This was because Snapchat had changed their website layout, but the user was unaware of it. They only noticed the issue after manually inspecting the output.\n\nThe user then investigated what had changed. Snapchat's public profiles (snapchat.com/@username) used to contain data within a script tag. However, this changed when the profile pages moved to the Next.js App Router. The script tag (__NEXT_DATA__) was no longer present. When the scraper encountered this change, it raised an error, caught it, logged it, and moved on to the next URL. In this case, with only one URL to process, moving on meant finishing with zero items and a successful exit code. This demonstrated the importance of checking the actual data rather than just the run status.\n\nThe user found that the data had moved to the HTML of the page, where it was streamed as React Server Components. They discovered that the video data was in the HTML as normal JSON objects, which could be extracted by searching for a marker and decoding one JSON value at a time using Python's json.JSONDecoder().raw_decode. This method allowed them to ignore any additional content that followed the JSON value.\n\nAnother bug was discovered when the user noticed that some Spotlight videos had empty caption files, but the audio was still present and transcribed by Whisper without issues. The user realized that an empty caption file did not necessarily mean silence and suggested treating both empty and missing caption files the same way, allowing Whisper to make the final decision on whether there was actually speech or not.",
  "summary": "On Monday I checked my Snapchat Spotlight scraper and it looked fine. Green status, run finished in five seconds, exit code 0. The dataset was empty. It had been empty since Snapchat pushed a change to their website, and nothing told me. No error, no failed run. If I hadn't opened the output by hand I would have kept \"selling\" a tool that returned nothing. Here's what broke, how I found the data…",
  "key_points": [
    "Snapchat's website layout changed, breaking the scraper",
    "Data moved to HTML of pages, streamed as React Server Components",
    "User discovered method to extract JSON video data from HTML"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}