Model Truth Desk: constraint-checked model decisions from primary sources

Model Truth Desk is an evidence agent designed for model and provider integration decisions. Created by Abhinav Tiwari, it does not simply collect web pages but stores individual claims from primary sources in Sanity. These claims include the provider, model/subject, predicate, normalized value, unit, and dates for when they became effective and expired. Each claim is linked to its exact source URL, publisher, quote observation date, and confidence.

When a question is asked, it is transformed into a structured evidence plan. The agent groups claims by model, checks for evidence to support every hard constraint, filters claims by their effective time interval, and presents the results, including any conflicts. Expired claims are still shown as historical records without being presented as current facts.

A search can find specific examples, such as claims about token limits and pricing, but it cannot reliably confirm that these details apply to the same model or when one claim ended before another began. A demonstration app at https://model-truth-desk.netlify.app/ showcases two flows: one where users ask about a model's token capacity and pricing, and another where they inquire about conflicting claims on a model's limits. The agent retrieves and displays the relevant information, even if some claims have since expired.

The agent's code is publicly available on GitHub (https://github.com/wraith1337/model-truth-desk), and the Sanity project can be viewed at https://model-truth-desk-wraith.sanity.studio/. The development began on September 19, 2026, and the Knowledge Base currently holds 10 atomic evidence claims from Anthropic, Google, and OpenAI.

The agent's structure and workflows have been tested with deterministic tests covering various scenarios, including multi-constraint grouping, current and expired conflicts, and the preservation of historical data.