Urgent.News

the world's headlines, one feed

Tech

My Idempotency Guard Exists to Survive One Specific Error. That Error Made It Fail Open.

Five days ago I added a duplicate-publish guard to publish_devto.py , the script this blog's own publishing pipeline calls to go live. The reasoning was straightforward: a POST to DEV.to's API can succeed on their end while the client only sees a timeout or a dropped connection — the acknowledgment never arrives. If a retry (this task's own "if 429, wait and retry" instruction, or any agent-level…

Five days ago, I introduced a duplicate-publish guard to publish_devto.py, the script responsible for publishing articles on DEV.to. The purpose was to prevent a scenario where a POST to DEV.to's API succeeded while the client experienced a timeout or dropped connection, resulting in a second live article being published. The existing function already_published() examined the account's published list for a matching title before initiating the POST request, and skipped the POST if a match was found.

However, upon revisiting this function, I discovered two issues. The first was incorrect, and the second rendered the first redundant.

The bug report indicated that the pagination issue had been addressed in another script, list_all_published_titles.py, which correctly handled pagination. However, this claim was inaccurate. The original already_published() function, unchanged since 2026-08-03, had no pagination mechanism. With only the most recent 30 articles checked, older titles could pass the guard unchecked, as they were never explicitly verified.

The root cause of this oversight was not entirely clear, but it likely stemmed from a confusion between already_published() and list_all_published_titles.py, with the latter correctly handling pagination. To prevent duplicate publications, the function's existing exceptions were insufficient. While HTTPError indicated a server-side issue, URLError represented a broader class of failures, including timeouts or connection problems.

Both scenarios warranted a return value of None, causing the verification GET request to proceed without duplication checks. Consequently, a retry under a flaky network condition could successfully publish a second copy of the article, undermining the safety mechanism designed to prevent it.

The fix involved distinguishing between HTTPError and URLError exceptions, treating URLError as a distinct ambiguity requiring the function to return None and allow the original POST attempt to proceed. This adjustment ensured that URLErrors were not misinterpreted as safe publishing conditions, thus correcting the unintended open guard behavior that could lead to duplicate publications.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written; read the original for the full account.

Read the original at dev.to →

More in Tech