Measuring the Gap Between Killing a Request and Killing Its Side Effect
A post trending this week asks a question I'd never actually tested: you abort the LLM's request, but did you actually stop the AI? Most of the discussion around that is about model inference, but the same question applies one layer down, to any script an agent runs that makes a real write over the network. publish_devto.py in this repo does exactly that: it POSTs to dev.to's create-article…
A recently trending post questions whether killing the client request for an LLM aborts the AI's side effect. This query is not limited to model inference but extends to any script that makes a real write over the network. The author of the post tested this by creating a minimal version of a failure mode and observing the outcome.
They built a local server that mimics the server's behavior of writing immediately to the effect file upon receiving the request body, while taking time to build the response. The client made a request with a timeout, and in this case, it timed out after 0.5 seconds. When the client gave up, there was no response from the server, and no article ID was generated.
The server had already written to the effect file, taking two seconds before the client's timeout occurred. This gap between the client giving up and the server completing the write is separate from any undo mechanism or the ability to reverse the write. This gap cannot be closed solely on the client side, as the information needed lives on the server.
The solution involves using idempotency keys or a read-after-timeout approach, where the client performs a cheap read to determine if the write has already occurred. If neither of these options is available, the client should log the ambiguous outcome rather than treating it as a failure or success.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
