I built the same agent in Strands, LangGraph, and CrewAI — and recorded every LLM call to see how they actually differ
Which framework should I actually use? Every comparison article has an opinion. Almost none of them has data. So I built the same agent three times — once in Strands , once in LangGraph , once in CrewAI — ran 27 executions, and routed every single LLM call through a local recorder proxy so the logs are directly comparable. The frameworks write different log formats, different trace shapes,…
This investigative article details a comprehensive comparison of three agent frameworks—Strands, LangGraph, and CrewAI—by implementing the same tech-news digest agent in each of them. The author then meticulously ran 27 executions and recorded every LLM call to analyze how the frameworks handled the process differently.
The frameworks exhibit distinct philosophies that translate into implementation differences. Strands operates in a model-driven manner, where the model autonomously decides tool usage, order, and when to stop. With only 78 lines of code, it allows for adaptive LLM calls based on the task at hand. In contrast, LangGraph employs a graph-driven approach, where the model only decides within nodes of a predefined state and edge structure.
This approach mandates a more deterministic workflow. CrewAI takes a role-based framework, where agents with defined roles and backstories work together through an orchestrator called Crew.
A crucial aspect of this study was the consistent use of a single model behind a local proxy that recorded every LLM call. This proxy abstracted away the different log formats and trace shapes produced by each framework, enabling a direct comparison of tokens used and latency. The findings revealed that LangGraph's explicit verify/revise loop significantly reduced output variance by 77% compared to Strands, but at the cost of 2.5x more tokens and latency.
The repeatability of CrewAI's output, driven by the use of temperature=0 and a role prompt, led to a fixed call structure with four calls in base mode, regardless of changes like a tighter word count requirement.
The analysis also sheds light on how each framework handles schema drift, where a minor change in the tool arguments (such as renaming "text" to "content") was smoothly absorbed by all three frameworks, with CrewAI being the most consistent in its response. Ultimately, the study highlights that while model-driven frameworks like Strands offer flexibility and adaptive reasoning, they come at the cost of variability in output and higher resource consumption.
In contrast, graph-driven and role-based frameworks provide more deterministic and consistent behavior but require more upfront setup and potentially less adaptability.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
