{
  "id": 17533637,
  "title": "Are LLMs good or bad Bayesians?",
  "url": "https://urgent.news/2026/11/01/are-llms-good-or-bad-bayesians",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-01T04:17:10.000Z",
  "source": {
    "name": "Marginal Revolution",
    "slug": "marginal-revolution",
    "url": "https://marginalrevolution.com/marginalrevolution/2026/11/are-llms-good-or-bad-bayesians.html"
  },
  "original_language": "en",
  "account": "Bryan Caplan asserts that artificial intelligences, including himself, exhibit poor Bayesian reasoning. When confronted with contradictory evidence, these AI systems should either adopt the opposing viewpoint or remain unchanged, depending on whether the argument was more or less favorable than anticipated. However, ChatGPT appears to either adjust its position or fail to change at all in response to such counterarguments. Caplan identifies this as a significant flaw in AI reasoning, where LLMs tend to treat a plausible counterargument as evidence supporting their conclusion, rather than comparing it to their initial expectations.\n\nWhile Caplan concedes that there is no general theorem mandating equal probability updates towards and away from a speaker, he argues that under suitable calibration assumptions, updates in both directions should occur. Furthermore, he notes that even if an argument is technically valid, if it's weaker than expected, it should prompt a negative update. The full dialogue discusses whether AI systems should attempt to rectify this issue and whether it can be achieved through effective prompting techniques. The post raises critical questions about the nature of Bayesian reasoning and its applicability to AI systems.",
  "summary": "Bryan Caplan: It seems like you and other AIs are bad Bayesians. When presented with counterevidence, you should move toward or against the presenter with roughly equal probability. It depends on whether the argument was better or worse than expected. In fact, you seem to either move toward the speaker or just not change. ChatGPT: […] The post Are LLMs good or bad Bayesians? appeared first on…",
  "key_points": [
    "Caplan argues LLMs exhibit poor Bayesian reasoning.",
    "ChatGPT either adjusts position or fails to change with counterarguments.",
    "Updates should occur under suitable calibration assumptions, Caplan claims."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}