MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places
We introduce MAP, the first benchmark to evaluate multimodal AI systems as assistants for users with accessibility requirements when planning visits to places in the real world. In our evaluation, systems are presented with requests to verify or recommend a point of interest meeting an accessibility requirement. MAP contains two novel assessments: Claim verification for accessibility planning…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
