{
  "id": 12583088,
  "title": "Filtering a 2 GB CSV in the browser without uploading it: streams, a Web Worker and a 64-bit hash set",
  "url": "https://urgent.news/2026/10/07/filtering-a-2-gb-csv-in-the-browser-without-uploading-it-streams-a",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-07T08:24:36.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/kamelyoul/filtering-a-2-gb-csv-in-the-browser-without-uploading-it-streams-a-web-worker-and-a-64-bit-hash-3mfp"
  },
  "original_language": "en",
  "account": "The architecture of CSV Géant is simple. The main thread handles the user interface and sends the File object to a Web Worker via postMessage. The Web Worker reads the file in 8 MB chunks, decodes and parses the CSV data, filters out duplicates, and writes the output to an in-memory Blob or files in a user-specified folder using the File System Access API. A Cloudflare Pages Function provides three routes for anonymous counters, license checks, and purchasing the tool. The code is lightweight and runs both in the browser and Node.",
  "summary": "Excel stops at 1,048,576 rows. Most online CSV tools want you to upload the file first, which is a non-starter for a customer list or an accounting export. I wanted something in between: open a multi-gigabyte CSV in a normal browser tab, filter it, remove duplicates, split it into Excel-sized files, and never send a byte of it anywhere . The result is CSV Géant , a static site with no framework…",
  "key_points": [
    "Main thread sends File object to Web Worker via postMessage",
    "Web Worker reads file in 8 MB chunks, filters duplicates",
    "Cloudflare Pages Function provides routes for counters, license checks, and purchasing"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}