{
  "id": 18978633,
  "title": "GPT-6 Astra Passed 69.9% of Hidden Checks. It Fully Finished 26.4% of Real Edit Tasks",
  "url": "https://urgent.news/2026/11/08/gpt-6-astra-passed-69-9-of-hidden-checks-it-fully-finished-26-4-of",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-08T17:01:37.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/gabrielanhaia/gpt-6-astra-passed-699-of-hidden-checks-it-fully-finished-264-of-real-edit-tasks-ki9"
  },
  "original_language": "en",
  "account": "GPT-6 Astra achieved an impressive 69.9% success rate in hidden acceptance checks for each task, with 26.4% of tasks fully passing. These results come from the Co-Design Bench paper submitted to arXiv on October 26 by researchers from Alibaba and various Asian universities. The benchmark tested ten model endpoints, with GPT-6 Astra emerging as the top performer. The remaining nine models achieved completion rates ranging from 56.7% to 51.1%. The hidden acceptance criteria, which the models never see during execution, are only revealed during grading, preventing reliance on a checklist. Each task provided a web app and a change request, with 39,325 production requests across 26,314 coding sessions used for testing. The grading process involved AI judges evaluating source diffs, rendered outputs, and browser traces using Playwright, with a strict score count only when a criterion was awarded a Pass.",
  "summary": "Book: Ship It — the pragmatic startup tech stack Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub Hand GPT-6 Astra a working web app and a real user's change request, and it clears an average of 69.9% of…",
  "key_points": [
    "GPT-6 Astra achieved 69.9% success in hidden acceptance checks",
    "26.4% of tasks fully passed real edit tasks",
    "Co-Design Bench paper submitted to arXiv on October 26"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}