Filtering a 2 GB CSV in the browser without uploading it: streams, a Web Worker and a 64-bit hash set
Excel stops at 1,048,576 rows. Most online CSV tools want you to upload the file first, which is a non-starter for a customer list or an accounting export. I wanted something in between: open a multi-gigabyte CSV in a normal browser tab, filter it, remove duplicates, split it into Excel-sized files, and never send a byte of it anywhere . The result is CSV Géant , a static site with no framework…
The architecture of CSV Géant is simple. The main thread handles the user interface and sends the File object to a Web Worker via postMessage. The Web Worker reads the file in 8 MB chunks, decodes and parses the CSV data, filters out duplicates, and writes the output to an in-memory Blob or files in a user-specified folder using the File System Access API.
A Cloudflare Pages Function provides three routes for anonymous counters, license checks, and purchasing the tool. The code is lightweight and runs both in the browser and Node.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
