Failed Webhook Backoff and DLQ Redrive for 3 Marketplace Deadline Guarantees

A marketplace must deliver reminders within a specific deadline to avoid failures. When an attempt cannot complete before the deadline, the reminder should go to the dead-letter queue. Proper handling involves using at-least-once delivery, stable idempotency keys, exponential backoff with jitter, and explicit redrive. A queue can accept all jobs, but if retries occur after the window closes, the business requirement will still be missed.

Exponential backoff with jitter prevents synchronized failure waves and controls pressure, but it doesn't guarantee delivery. The business deadline is the contract, not the number of retries. Each successful reminder should have a durable delivery ID, and the consumer must treat it idempotently. The scheduler won't initiate new automatic attempts if the estimated completion time surpasses the business deadline.

Just like in exact-once delivery scenarios, at-least-once delivery with receiver-side idempotency is a safer choice. The message should carry both business and delivery state separately, and the deadline should always be included. The visibility timeout is an operational tool, not proof of message processing. Acknowledge only after classifying the result and persisting the state transition.

If the process dies between the remote commit and the local acknowledgement, another worker will attempt the same delivery ID, handling duplicates. The deadline should be managed as a state machine, classifying response types, and treating retryable errors appropriately. Authentication failures should be quarantined and escalated.

Here's a simple Python model to manage the scheduling decision. It's a pure function, making it easier to test the boundary conditions. This model can be used in any worker language, as long as the same fields and transition rules are applied.