Do agents survive a crash, and why does an LLM retry execute the same side effect twice? 34 runs measured
When you hand an agent work with external effects, two worries always come up. First: when the process dies, what happens to the work in progress? An agent paused at an approval gate crashes — can it resume, or does everything restart from scratch? Second: when an LLM retries, does it execute the same side effect twice? Retrying a send or a data mutation carries a real double-execution risk. My…
The article examines the behavior of agents with external effects, specifically when an agent crashes and when an LLM retries a side effect. Three frameworks were tested—LangGraph, Strands, and CrewAI—in three cells: crash recovery, idempotency, and audit under retry. Each framework was run multiple times with the same task, model, and recorder proxy.
In the crash recovery cell, LangGraph's durable checkpointer allowed for a quick resume (0.01-0.02s) compared to full re-runs (Strands, CrewAI) averaging 4.2-4.9 seconds. Without a checkpointer, Strands and CrewAI performed full re-runs. If the process dies before the first checkpoint is persisted, LangGraph may fail to resume (version-dependent).
The idempotency cell investigated three key strategies: position key (intent-based), content hash (raw arguments), and no key. The content hash key silently allowed retries to execute side effects twice when the model reworded the arguments. The position key identified the intent and ensured the same key was not reused even if arguments changed. The no key strategy did nothing to prevent double execution.
In the audit under retry cell, an auditor examining traces alone could detect duplicates 100% of the time across all cells, proving the deduplication. However, this was due to the recording being at the wire level, with the proxy keeping all attempts as separate records. The framework-level traces did not show the double firing. The article concludes that silent failures, such as the content hash key's behavior, can lead to double execution, making it the riskiest approach for enterprise use.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
