SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving
We introduce SWE-Serve, a benchmark for evaluating agents on production inference engineering tasks. Implementing an inference feature can require coordinating multiple changes across the serving stack, including model support, runtime execution, and public APIs. Existing benchmarks provide limited coverage of production inference engineering: repository-level software engineering benchmarks do…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
