CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?
Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a local plugin change can propagate through dependencies and cleanup. We introduce CordisBench, a 1,200-question benchmark of this lifecycle reasoning. It combines a controlled formal setting with programs executed against Cordis, a runtime that manages…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.