Urgent.News

What's breaking now, across thousands of outlets.

Tech

Practical Handling of Webhook Task Retries After Failure in Delayed Queue Node.js Systems

Short answer: handle failed webhook task retries with a durable event record, delayed exponential backoff, a leased worker, and receiver-side idempotency. Treat delivery as at-least-once, then make duplicate attempts harmless. A failed webhook task needs two separate decisions: when to try again, and how to make a repeated attempt harmless. The queue can provide at-least-once work delivery. It…

Handling webhook task retries in delayed Node.js systems requires a systematic approach. First, acknowledge that delivery should be at-least-once to avoid loss, but design operations to tolerate repetition. Identify two critical decisions: when to retry and how to make repeated attempts harmless.

Implement a durable event record to track each webhook task. Include an event ID, endpoint, payload version, attempt count, next eligible time, last error class, and terminal status in the record. The event ID remains constant across attempts, while the attempt ID changes. This allows the receiver to treat the ID as a deduplication key, ensuring duplicates are controlled.

Adopt exponential backoff with a cap for retries. After each attempt, calculate the delay using a formula that increases exponentially but caps at a maximum value. Add a random jitter to the delay to avoid thundering herd problems. Limit the maximum number of attempts to prevent indefinite retries.

Use a leased worker to prevent two workers from processing the same webhook task simultaneously. The worker acquires a lease for a limited time (e.g., 30,000 milliseconds). If the lease is not granted, the task is marked as dead. If the retry is successful, mark the task as delivered. If the retry fails, schedule it for a later attempt based on the calculated delay.

Store the delivery state and idempotency rules within the application to keep the mechanism simple and maintainable. A small worker that preserves the boundary between retries and delivery states is sufficient. The worker should not delete tasks until the delivery state has been updated, ensuring visibility of the failure sequence.

By focusing on these key elements, webhook task retries can be handled effectively, ensuring eventual delivery while preventing duplicate operations.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

The Rule Was on the Other Branch

Every team I have worked with keeps its practices in a dozen places. The PR checklist lives in one repository. Who promotes to production is on a wiki page for the whole organisation.

oviev: Building a Compiler in Ovie The Self-Hosting Journey

One of the milestones of any programming language is when it can compile itself. Ovie Lang 1.0 ships oviev the Ovie compiler written entirely in Ovie (.ov files).

  • Ovie compiler oviev written entirely in Ovie
  • oviev resides in oviev/src/ directory with .ov files in oviec/src/
  • Stage 1 compiler delegates to Rust-based oviec for compilation

More from Tuesday 17 November →