Reliable Webhooks in TypeScript: Idempotency, Deduplication and Failure Recovery

Webhook providers typically send messages more than once, not exactly once. This fact is the root cause of many webhook bugs, such as duplicate emails, double charges, and orders being marked paid twice.

To safely handle webhook deliveries, there are three key properties to implement: authenticity, idempotency, and recoverability. Here is a step-by-step guide on how to achieve this using a PostgreSQL table and TypeScript.

1. Verify the signature: Signature verification should use the exact bytes the provider signed, not the JSON-reserialized version. Import the crypto module and create a function to compare the HMAC digest of the raw body with the expected digest. Use a constant-time comparison and reject stale timestamps if the provider provides one.

2. Record the event before acting on it: Create a table with a unique constraint on the provider's event ID. Insert the event data with ON CONFLICT DO NOTHING to ensure deduplication works correctly even with concurrent deliveries. Acknowledge the provider in case the event was seen before, returning a 200 status code.

3. Separate acknowledging from processing: Providers may time out slow endpoints and retry. Acknowledge the event quickly by returning a 2xx status code, and process the event asynchronously. Use a simple worker that polls the webhook_events table for events in the 'received' status and claim them with a row lock to prevent multiple workers from processing the same event.

4. Make the business effect idempotent: Deduplicating the event is not enough, since the same real-world action can arrive under different event IDs. Use a natural key, such as a provider's payment ID, to anchor the effect to a single, naturally unique piece of data. Update the state and the status = processed simultaneously in the same transaction to ensure idempotent behavior.