Count-Min Sketch: counting millions of events with kilobytes of RAM
Un firewall que ve millones de paquetes por segundo no puede abrir una entrada en una tabla hash por cada IP distinta que procesa: se queda sin memoria en minutos. Necesita contar sin guardar cada evento, y ahí entra el count-min sketch , una estructura de datos probabilística que estima cuántas veces ocurrió algo usando apenas unos kilobytes. El count-min sketch resuelve ese problema con una…
The count-min sketch is a probabilistic data structure that estimates the frequency of events in large datasets without storing each individual event. It was introduced by Graham Cormode and S. Muthukrishnan in 2005 as a solution to counting frequencies in data streams that are too large to fit in memory. The structure uses a small matrix and hash functions to provide an estimate with bounded error and constant memory usage.
It is widely used in streaming systems, analytical databases, and network monitoring, and is particularly useful for detecting heavy hitters in network traffic and counting event frequencies in large datasets. Redis, Apache Flink, and other systems use the count-min sketch to count events at scale.
Written by urgent.news from Dev.to's report — not a translation of it. Machine-written — may contain errors; check the original before relying on it.
