When an AI model doubts evidence that was sound
We gave Claude 70 claims, each with the evidence offered for it. Unguided, it almost never passed weak proof, yet it turned down a third of the sound ones. Mostly by refusing evidence that was fine. We wrote 70 short cases, each a claim about a Cloudflare site, a payment on the Base chain or a web page, paired with the check someone ran to back it, then had Claude Sonnet and Claude Haiku decide…
We haven't written up this one. Dev.to has the full story — the link below goes straight to it.