GPT-6 Astra Passed 69.9% of Hidden Checks. It Fully Finished 26.4% of Real Edit Tasks
Book: Ship It — the pragmatic startup tech stack Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub Hand GPT-6 Astra a working web app and a real user's change request, and it clears an average of 69.9% of…
GPT-6 Astra achieved an impressive 69.9% success rate in hidden acceptance checks for each task, with 26.4% of tasks fully passing. These results come from the Co-Design Bench paper submitted to arXiv on October 26 by researchers from Alibaba and various Asian universities. The benchmark tested ten model endpoints, with GPT-6 Astra emerging as the top performer.
The remaining nine models achieved completion rates ranging from 56.7% to 51.1%. The hidden acceptance criteria, which the models never see during execution, are only revealed during grading, preventing reliance on a checklist. Each task provided a web app and a change request, with 39,325 production requests across 26,314 coding sessions used for testing.
The grading process involved AI judges evaluating source diffs, rendered outputs, and browser traces using Playwright, with a strict score count only when a criterion was awarded a Pass.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
