SABRE: Scalable and Automated Benchmarking of VLMs under Stress
Vision-language models (VLMs) are improving rapidly, but benchmark development lags behind, making weaknesses hard to identify. Building stress tests is costly: samples must satisfy controlled conditions, remain answerable, and challenge current models. We present SABRE, a scalable, automated pipeline that converts a Test Primer (a Markdown Task Design with Data Schema) into structured…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.