SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories
Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks, however, usually start from a human-identified issue and evaluate whether a patch satisfies a functional signal. We present SWE-Prometheus, a benchmark for the broader task of improving repository engineering governance. Each task provides a fixed…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
