Urgent.News

What's breaking now, across thousands of outlets.

Tech

Prevent toxic messages from freezing your pipeline: Retries and DLQ.

Evita que los mensajes tóxicos congelen tu pipeline: Reintentos y DLQ. Día 08 de la serie técnica WKafka Open Source. Un único JSON malformado jamás debería tumbar un consumidor de Kafka en producción. WKafka aísla los mensajes tóxicos en una Dead Letter Queue automáticamente. Los Problemas Reales Un único payload corrupto congelando la partición en bucles de crash infinitos Tormentas de…

Translated from Spanish Read in Spanish

WKafka, an open-source project, has released the 8th technical article in its series, focusing on preventing toxic messages from freezing a pipeline. The issue arises when a single malformed JSON causes a Kafka consumer to crash in production. WKafka addresses this by automatically isolating problematic messages in a Dead Letter Queue (DLQ).

This implementation allows for automatic routing of permanent failures to a designated DLQ topic, exponential backoff to prevent request storms, and zero blocking to ensure healthy messages continue processing. WKafka is compatible with Python 3.9 to 3.14 and is available on GitHub and PyPI.

Written by urgent.news from Dev.to's report — not a translation of it. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

Stop poison pills from halting your Kafka pipeline: Retries and DLQ.

Stop poison pills from halting your Kafka pipeline: Retries and DLQ. Day 08 of the WKafka Open-Source Engineering Series.

  • Implement retries and DLQ to prevent Kafka pipeline halts
  • Kafka automatically isolates poison pills into DLQ
  • Exponential backoff prevents thundering herds on recovery

Enterprise-level high availability in Redis: Sentinel and Cluster with WRedis

Alta disponibilidad de nivel empresarial en Redis: Sentinel y Cluster con WRedis Día 08 de la serie técnica Wisrovi Open Source Architecture.

  • Redis Sentinel monitors node health and performs automated failover
  • Redis Cluster enables horizontal distribution and automatic redirection
  • Configurations use SentinelRedisManager and ClusterRedisManager classes

We checked 53,709 production resources for backups. 61% had none.

Disclosure: I work at Eon, on the content side. Our team ran these assessments and this is our data. I didn't write the queries, but I sat with the engineer who did, and the methodology below includes what we couldn't see. Argue with it. Most backup tools produce a coverage report, and most of those reports work from tags.

More from Monday 28 September →