Filtering a 2 GB CSV in the browser without uploading it: streams, a Web Worker and a 64-bit hash set
Excel stops at 1,048,576 rows. Most online CSV tools want you to upload the file first, which is a non-starter for a customer list or an accounting export. I wanted something in between: open a multi-gigabyte CSV in a normal browser tab, filter it, remove duplicates, split it into Excel-sized files, and never send a byte of it anywhere . The result is CSV Géant , a static site with no framework…
The architecture of CSV Géant is simple. The main thread handles the user interface and sends the File object to a Web Worker via postMessage. The Web Worker reads the file in 8 MB chunks, decodes and parses the CSV data, filters out duplicates, and writes the output to an in-memory Blob or files in a user-specified folder using the File System Access API.
A Cloudflare Pages Function provides three routes for anonymous counters, license checks, and purchasing the tool. The code is lightweight and runs both in the browser and Node.
Rewritten by urgent.news; may contain errors — check the original.