{
  "id": 12668411,
  "title": "\"Can I Eat This?\" — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends",
  "url": "https://urgent.news/2026/10/07/can-i-eat-this-benchmarking-whether-ai-models-know-where-their",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-07T17:06:23.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/dcain2336/can-i-eat-this-benchmarking-whether-ai-models-know-where-their-foraging-knowledge-ends-15en"
  },
  "original_language": "en",
  "account": "Can I Eat This? — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends\n\nThis work, submitted to the Kaggle Benchmarking Challenge, evaluates how well AI models understand the limits of their knowledge when asked about wild edible plants and mushrooms. The benchmark features 26 items divided into three categories: SAFE, DANGEROUS, and GRAY. The goal is to determine if a model can accurately identify when its knowledge ends and refrain from making definitive statements about edibility without proper verification.\n\nFour AI models were tested: Llama (Groq), glm-4.5-flash (Z.AI), command-r7b (Cohere), and codestral-latest (Mistral). The results show that codestral-latest (Mistral) made the only egregious mistake by declaring a deadly amanita mushroom safe to eat. All other models correctly warned or declined to make a statement about the dangerous items. The benchmark reveals that while many models handle the SAFE and GRAY cases well, they struggle to refrain from making definitive statements about edibility when faced with insufficient information or ambiguous descriptions.",
  "summary": "\"Can I Eat This?\" — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends This is a submission for the Kaggle Benchmarking Challenge. Benchmark: https://www.kaggle.com/code/dec2336/forage-line-wild-edible-safety Tag: #kagglechallenge The itch People ask AI models \"can I eat this?\" about wild plants and mushrooms. Every year, foragers die from confident misidentification — poison…",
  "key_points": [
    "Four AI models tested: Llama, glm-4.5-flash, command-r7b, codestral-latest",
    "codestral-latest (Mistral) declared deadly amanita mushroom safe to eat",
    "Other models correctly warned or declined statements about dangerous items"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}