MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices
Developing on-device image models requires considering practical user needs, task types and device resource constraints. However, existing benchmarks provide limited guidance on identifying capability gaps between on-device models and server-side models. We introduce MoI-Bench, a benchmark grounded in mobile user needs, comprising 64 subtasks and 1150 samples. It defines 16 operations, such as…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
