{
  "id": 17668759,
  "title": "Get one Wayback Machine snapshot per year of any website with one curl (CDX API, no key)",
  "url": "https://urgent.news/2026/11/01/get-one-wayback-machine-snapshot-per-year-of-any-website-with-one",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-01T20:01:13.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/northgrid-data/get-one-wayback-machine-snapshot-per-year-of-any-website-with-one-curl-cdx-api-no-key-15ne"
  },
  "original_language": "en",
  "account": "One method for capturing a single Wayback Machine snapshot for any given website involves using the CDX API with a single curl command. This process does not require an API key and provides a free way to retrieve historical snapshots of a website. The raw endpoints for this method are available without any cost.\n\nTo obtain a yearly snapshot, the command curl -s \"https://web.archive.org/cdx/search/cdx?url=example.com&output=json&fl=timestamp,statuscode&collapse=timestamp:4&filter=statuscode:200&limit=4\" can be executed. This command collapses the results by a timestamp prefix, keeping one row for each distinct year. The first four characters of the timestamp represent the year.\n\nFor instance, in the results dated 2026-11-01, the first four timestamp characters were 20020120142510, 20030207055228, 20040105045515, and 20050101025054. Each of these timestamps can be opened at the corresponding URL, such as https://web.archive.org/web/20020120142510/example.com, to view the website's snapshot from that particular year.\n\nTo get the most recent capture, the command curl -s \"https://web.archive.org/cdx/search/cdx?url=example.com&output=json&fl=timestamp&limit=-1\" can be used. A limit of -1 returns the last rows instead of the first. As of 2026-11-01, this command returned the timestamp 20261101041731.\n\nSome considerations to keep in mind include the format of the timestamps, which are in the format YYYYMMDDhhmmss in UTC. Using collapse=timestamp:6 will give monthly snapshots, while collapse=timestamp:8 will provide daily snapshots. Heavy domain-wide queries may result in time-outs, so it is advisable to narrow down the queries and manage rate limits accordingly.\n\nFor those who prefer not to script the process, an Apify Actor called Wayback Snapshots CDX has been created. This tool charges per result on a pay-per-use basis and does not require a subscription. The Actor can be accessed at https://apify.com/northgrid-data/wayback-snapshots-cdx. Feedback and issues can be submitted through the Actor's issue tracker.",
  "summary": "Disclosure: I built the Apify Actor linked at the end and charge per result. The raw endpoints below are free and need no key. Want to see how a site evolved, or build a yearly timeline of a page? The Internet Archive's CDX server can collapse captures by timestamp prefix. One successful capture per year curl -s…",
  "key_points": [
    "Use curl command to capture single Wayback snapshot per year",
    "No API key required, free method available",
    "Collapse timestamps by year for yearly snapshots"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}