Frontier Design Benchmark
Frontier Design Benchmark measures how capable models are at creating appealing frontend designs as compared against human designers that were given the same prompt. The surprise is that a stealth model is beating every public model and even human created datasets. Comments
Frontier Design Benchmark marks a significant step in assessing the capabilities of AI design models. This benchmark, based on real-world high-value and complex design projects, is evaluated by professional designers in Contra Labs network, ensuring the quality of assessment.
A unique feature of this benchmark is the design critique rubric and a list of AI slop patterns. These metrics, highly correlated with designers' ratings of the models (r = 0.94), provide explainable design quality judgment alongside pairwise preferences. The benchmark uses real client briefs, presenting each design set beside the models' outputs side by side.
This allows designers to directly compare two model outputs for the same brief without knowing the models' names. The choice to pick the better output or call a tie is left to the designer's discretion.
Written by urgent.news from Lobsters's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.