{
  "id": 13870740,
  "title": "Okie-dokie, Casey Handmer edition",
  "url": "https://urgent.news/2026/10/12/okie-dokie-casey-handmer-edition",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-12T04:20:47.000Z",
  "source": {
    "name": "Marginal Revolution",
    "slug": "marginal-revolution",
    "url": "https://marginalrevolution.com/marginalrevolution/2026/10/okie-dokie-casey-handmer-edition.html"
  },
  "original_language": "en",
  "account": "Casey Handmer is working on an automated peer review project using current models, which can provide human-level review for approximately $0.10 in tokens. By investing $5, one could rederive every result, verify all statistical inferences, and audit every paper cited in both directions. However, there are around 100 million papers in the scientific corpus, with most of them being unimportant or poorly executed. Due to structural limitations, correcting published work, even for typos, is nearly impossible. For $500 million, it would be possible to thoroughly audit every paper ever written, identifying p-hacked results, plagiarists, fabrications, and citation clubs. As Handmer cannot afford this amount, he is focusing on a tree approach, targeting the 1% of papers that hold 99% of the signal. He can find 90% of the nonsense for a few hundred dollars of tokens. The biggest surprises may be revealed in this process. The details can be found <here>. This post appeared on Marginal REVOLUTION.",
  "summary": "I’ve been prototyping an automated peer review project. With current models you get human level peer review for about $0.10 of tokens. For $5 you can rederive every result, check every statistical inference, and audit every paper cited in both directions. There are about 100m papers in the scientific corpus. Most are not important. Many […] The post Okie-dokie, Casey Handmer edition appeared…",
  "key_points": [
    "Casey Handmer develops automated peer review project",
    "$0.10 per token can provide human-level review",
    "Focusing on 1% of papers with 99% of signal"
  ],
  "editors_take": "Automating the peer review process for high-impact research promises to expose scientific misconduct and errors by making the auditing of critical papers significantly more affordable than manual verification.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}