Same Simulation, Different Evidence: Where Retrieved Tool Output Costs LLM Agents Accuracy
LLM agents that compute the right answer can still answer wrongly. Agent harnesses increasingly keep long tool outputs out of the context window, saving them to files that the agent searches or reads on demand. We ask where this costs accuracy with a real scientific tool: PHREEQC, a widely used geochemistry simulator from the USGS, whose reports reach 846.8k characters and whose determinism lets…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.