{
  "id": 18848697,
  "title": "Why Telling an LLM to Be Careful Doesn't Work: Penalty Prompts Had a 0% Abstention Rate",
  "url": "https://urgent.news/2026/11/08/why-telling-an-llm-to-be-careful-doesnt-work-penalty-prompts-had-a-0",
  "topic": "world",
  "section": "World",
  "published": "2026-11-08T03:31:56.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/fgrandon/why-telling-an-llm-to-be-careful-doesnt-work-penalty-prompts-had-a-0-abstention-rate-49ij"
  },
  "original_language": "en",
  "account": null,
  "summary": "A recent study conducted on Gemma 2, 3, and 4 models using Ollama found that penalty prompts are ineffective in making language models admit ignorance or abstain from answering when they are not completely sure. In ten controlled experiments, models exhibited an abstention rate of 0%, consistently bluffing with near-total confidence and hallucinating information. This phenomenon is attributed to the mathematical inevitability of Good-Turing missing mass, symmetric reinforcement learning rewards, and the greedy argmax bottleneck in autoregressive language models. The researchers concluded that prompt engineering is not an effective solution and instead recommend building physical, sub-15ms circuit breakers to prevent forced hallucinations in production agent architectures.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}