{
  "id": 14681533,
  "title": "Astra Held Its Ground (Truth)",
  "url": "https://urgent.news/2026/10/15/astra-held-its-ground-truth",
  "topic": "finance",
  "section": "Finance & Markets",
  "published": "2026-10-15T11:18:46.000Z",
  "source": {
    "name": "Marginal Revolution",
    "slug": "marginal-revolution",
    "url": "https://marginalrevolution.com/marginalrevolution/2026/10/astra-held-its-ground-truth.html"
  },
  "original_language": "en",
  "account": "TheoPhysEval has revealed a collection of 50 theoretical physics problems that were designed to be challenging for AI agents. Typically, a world expert could solve such a problem in under a day, but these issues proved to be significantly more difficult when tackled autonomously by AI without any human guidance. The best-performing AI agent was GPT-6-Astra, which was able to obtain the correct final answers with an impressive accuracy of 98% at an average cost of under $3 per problem.\n\nOne example problem from the collection involved constructing a two-parameter Hamiltonian using the suspension construction method for one-dimensional class D fermionic systems with no microscopic symmetries other than fermion parity. The task required identifying the central charge of the critical theory at the diabolical point and determining what the diabolical point destabilizes to under free-fermion perturbations.\n\nInterestingly, the researchers initially thought Astra's score was in the high eighties, second only to another agent named Opus 5.5. However, upon closer inspection, they discovered that Astra insisted its answers were correct, even when faced with the experts' feedback. In multiple instances, the AI pointed out subtle errors in the experts' calculations that had previously gone unnoticed. As the experts revised their answers, Astra's score began to climb, ultimately resulting in a perfect score of 98% (49/50).\n\nThis surprising turn of events led the authors to conclude that the AI's answers had become the definitive ground truth for the experts, rather than the other way around. The authors expressed their amazement at this development, dubbing it \"the fuck\" level of performance.",
  "summary": "TheoPhysEval: We present a suite of 50 unpublished research-level theoretical physics problems with verifiable answers. Without AI assistance, a typical problem in our collection could be solved by a world expert in about a day or less but would take considerably longer for experts in adjacent specialties. We test the ability of off-the-shelf AI agents […] The post Astra Held Its Ground (Truth)…",
  "key_points": [
    "Astra AI achieved 98% accuracy on 50 theoretical physics problems.",
    "GPT-6-Astra solved problems in under $3 average cost each.",
    "Astra's answers became definitive ground truth for experts."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}