Two-Layer Idempotency: Why Not Every Settlement System Needs an Outbox
A few weeks ago I published a write-up on inverting the control flow of a settlement API: instead of the endpoint owning window semantics (batching, cutoffs, retries), it became a pure write-and-forget operation, and all of that coordination logic moved to a cold-path scheduler. One of the best responses I got was from a fellow engineer who pushed on the exact point that matters most in any…
In a recent discussion about settlement APIs, a key concern was the event publishing between the point of transaction commit and the broker's acknowledgment. This uncertainty led to the proposal of a two-layer idempotency model as an alternative to the traditional outbox pattern. The outbox pattern, which writes events to a separate table and has a separate poller publish to the broker, is a reliable solution for most systems, but it comes with added complexity and operational overhead.
For a settlement system with a short settlement window, this overhead may not be justified. The two-layer idempotency model addresses the issue in a different way. The first layer ensures correctness of causality by publishing messages only when the transaction is committed, thus preventing phantom messages. The second layer handles duplicate delivery by computing a SHA-256 checksum of incoming messages and checking them against Redis.
If a message is a duplicate, it is discarded, turning at-least-once delivery into effectively-once processing. This approach is lightweight and does not require the same level of operational complexity as the outbox pattern. The two-layer idempotency model offers a balance between reliability and simplicity, making it a suitable choice for certain settlement systems.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
