From 100 Million Cells to One Molecule You Can Walk Inside — Why Cascade Makes NexoPeek Essential Now
From 100 Million Cells to One Molecule You Can Walk Inside — Why Cascade Makes NexoPeek Essential Now I still remember the night the data dump landed on my laptop. A single CSV, 100 million rows, each row a cell from a spatial transcriptomics experiment. The file was 27 GB, the kind of thing that makes a laptop fan sound like a jet engine. I had a deadline in two days, a grant review looming, and…
In the aftermath of a 100 million cell spatial transcriptomics experiment, the data dump proved daunting. A single CSV file, 27 gigabytes in size, threatened to overwhelm even the most powerful laptops. With a looming grant review deadline, the usual "load-into-pandas-and-plot" approach was rendered inadequate. A viral Hacker News post caught the eye, promising a solution.
The post introduced Cascade, a pipeline that divided the massive cellular atlas into hierarchical tiles, each small enough to stream to a browser without sacrificing resolution. NexoPeek, a web-based viewer built on Cascade, allowed users to "walk inside" the data, rotating, slicing, and measuring distances in real time. Intrigued, the reporter cloned the open-source Cascade repository, spun up a local MinIO bucket, and ran the ingestion script on a 4-core VM.
The ingestion process took three hours, producing 12 terabytes of tiled Parquet files and a manifest JSON that NexoPeek reads at load time. Upon opening the viewer in Chrome, the 100 million-cell dataset appeared as a faint cloud. As the reporter hovered, the tile loader fetched the next level, revealing individual nuclei. Further zooms exposed a single mRNA molecule, its coordinates precise to the nanometer.
The experience was visceral, the viewer never freezing, with a flat frame-time graph indicating smooth performance. This demonstrated the two critical aspects of cascade architecture: progressive refinement, ensuring the first tile seen is a coarse summary, and stateless streaming, allowing tiles to be immutable objects stored in object storage without the need for a persistent server session.
The reporter faced challenges such as Parquet writer limits and WebGPU shader compatibility issues, but each hurdle was overcome, highlighting the transparent nature of the stack, which facilitated debugging. The cascade design mirrors biological organization, from tissues to cells to molecules, and this hierarchical approach enables researchers to scale spatial data products without engineering complexities.
For those dealing with spatial data that seems too large for browsers, the cascade-tiling approach is recommended. The open-source repo, ingestion scripts, and a minimal NexoPeek starter are available on GitHub. By trying the demo on a small subset, users can explore the first tile and scale up, allowing the data to tell its story.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
