Estimación de cardinalidad hiper-eficiente: Redis HyperLogLog en producción con wredis
Estimación de cardinalidad hiper-eficiente: Redis HyperLogLog en producción con wredis Contar elementos únicos en millones de usuarios activos diarios, direcciones IP o eventos telemétricos IoT en una base de datos relacional tradicional o en un Set estándar de Redis agota rápidamente la memoria RAM disponible. Un Set en Redis almacenando 100 millones de UUIDs requiere varios gigabytes de RAM. En…
HyperLogLog Cardinality Estimation in Production with wredis: Redis HyperLogLog in Production with wredis
Counting unique elements in millions of daily active users, IP addresses, or IoT telemetry events overwhelms traditional relational databases or standard Redis Sets with RAM. Redis HyperLogLog (HLL) implements a probabilistic estimation algorithm that keeps memory consumption constant at ~12 KB per key, while maintaining an error standard below 0.81%.
Why HyperLogLog with wredis? wredis integrates cardinality operations with strict typing, automatic TTL management, and high-concurrency connection pooling.
```python
from wredis import RedisHLLManager
# Initialize HLL Manager with enterprise-level pooling
hll_manager = RedisHLLManager(
host='localhost',
port=6379,
db=0
)
# Register daily unique visitors with distributed keys
key_visitors_today = 'visitors:2026-09-20'
hll_manager.add(key_visitors_today, *[f'user_{i}' for i in range(100000)], ttl=86400)
# Massive ingestion of unique identifiers with sub-millisecond latency
lote_usuarios_1 = [f'user_{i}' for i in range(100000, 150000)]
hll_manager.add(key_visitors_today, *lote_usuarios_1, ttl=86400)
# Estimate total unique visitors with bounded memory (~12 KB)
total_uniques = hll_manager.count(key_visitors_today)
print(f'Estimated unique visitors: {total_uniques}')
# Expected result: ~150,000 unique records with 0.81% margin of error
```
Key advantages for production architecture:
Constant footprint: Exactly 12 KB per counter, regardless of 10,000 or 10,000,000 distinct records.
Ultra-fast merging: Combines multiple HyperLogLog keys (e.g., daily metrics into weekly or monthly accumulations) directly on Redis without transferring massive data to the application.
Built-in TTL and lifecycle control: Prevents the accumulation of outdated analytical metrics in memory.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
This story
This is one outlet's version. Read the fullest account.