SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents
Repository-level software engineering benchmarks have significantly advanced the evaluation of coding agents, but existing benchmarks primarily measure whether generated patches pass functional tests and overlook review-derived acceptance constraints (review constraints) that often influence whether a patch is acceptable in real-world software development. We introduce SWE-Gate, a…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
