Urgent.News

What's breaking now, across hundreds of outlets.

Tech

Filtering a 2 GB CSV in the browser without uploading it: streams, a Web Worker and a 64-bit hash set

Excel stops at 1,048,576 rows. Most online CSV tools want you to upload the file first, which is a non-starter for a customer list or an accounting export. I wanted something in between: open a multi-gigabyte CSV in a normal browser tab, filter it, remove duplicates, split it into Excel-sized files, and never send a byte of it anywhere . The result is CSV Géant , a static site with no framework…

The architecture of CSV Géant is simple. The main thread handles the user interface and sends the File object to a Web Worker via postMessage. The Web Worker reads the file in 8 MB chunks, decodes and parses the CSV data, filters out duplicates, and writes the output to an in-memory Blob or files in a user-specified folder using the File System Access API.

A Cloudflare Pages Function provides three routes for anonymous counters, license checks, and purchasing the tool. The code is lightweight and runs both in the browser and Node.

Rewritten by urgent.news; may contain errors — check the original.

Read the original at dev.to →

Report a problem with this story

More in Tech

More from Wednesday 7 October →