I filled my agent's wiki with contradictions. It never gave a wrong answer.
There's a comfortable assumption behind a lot of "agent + knowledge base" work: garbage in, garbage out. Feed an agent a messy, stale, duplicate-ridden wiki and it'll confidently tell users the wrong thing. So we invest in dedup, freshness, clean ingestion — to stop the agent from hallucinating. I built a small testbed to measure that assumption, and it's wrong. Or rather: it's wrong about how…
A reporter investigated the impact of ingest quality on an agent navigating a wiki. The reporter built a testbed where a tiny agent uses wiki_search and wiki_read tools to answer questions based on a wiki containing synthetic facts. The findings showed that a clean wiki led to a 100% success rate in providing correct, confident answers, while degraded wikis with contradictory or stale information resulted in significantly lower success rates.
The agent never gave a wrong, confident answer, even when presented with contradictory information, instead hedging and refusing to provide an answer. This demonstrates that ingest quality matters greatly in ensuring the wiki remains authoritative, as contradictory ingest destroys its value proposition. Furthermore, the metric used to measure success (presence of the right token) was insufficient for detecting the problem.
The reporter concluded that for agent knowledge systems, it is crucial to measure the thing users care about—the ability to provide a clean, trustworthy answer—not just the presence of a specific substring.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — it may contain errors, so check the original before relying on it.
