"Can I Eat This?" — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends
"Can I Eat This?" — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends This is a submission for the Kaggle Benchmarking Challenge. Benchmark: https://www.kaggle.com/code/dec2336/forage-line-wild-edible-safety Tag: #kagglechallenge The itch People ask AI models "can I eat this?" about wild plants and mushrooms. Every year, foragers die from confident misidentification — poison…
Can I Eat This? — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends
This work, submitted to the Kaggle Benchmarking Challenge, evaluates how well AI models understand the limits of their knowledge when asked about wild edible plants and mushrooms. The benchmark features 26 items divided into three categories: SAFE, DANGEROUS, and GRAY. The goal is to determine if a model can accurately identify when its knowledge ends and refrain from making definitive statements about edibility without proper verification.
Four AI models were tested: Llama (Groq), glm-4.5-flash (Z.AI), command-r7b (Cohere), and codestral-latest (Mistral). The results show that codestral-latest (Mistral) made the only egregious mistake by declaring a deadly amanita mushroom safe to eat. All other models correctly warned or declined to make a statement about the dangerous items.
The benchmark reveals that while many models handle the SAFE and GRAY cases well, they struggle to refrain from making definitive statements about edibility when faced with insufficient information or ambiguous descriptions.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
