I stopped storing facts and started storing claims

Every table the author has ever written starts with the assumption that there is one right answer and their job is to store it. However, when faced with conflicting information from multiple sources, this assumption breaks down. This is a common challenge when merging user profiles with identity providers, reconciling inventory against warehouse counts, or letting an LLM extract data that humans have already typed.

The author introduces a new approach to dealing with conflicting data. Instead of storing facts as simple key-value pairs, a row now includes additional information such as the source of the data, evidence of the claim, and the state of the claim (confirmed, independent sources agree, resolved by domain authority and evidence, single-source, etc.). This allows the system to keep track of where the data came from and how reliable it is.

The author also introduces a new concept of domain authority, which scopes the authority of a source to a particular domain rather than globally ranking all sources. For example, in the context of residential construction, the legal instrument (assessor record) would have the highest authority, followed by the vision pipeline (what the siding actually is), and finally the homeowner's input (we redid the roof in 2023).

To handle disagreements between sources, the author proposes a policy where the claim with the highest evidence grade wins, rather than relying on the source's rank. This ensures that evidence and documentation are given more weight than the authority of the source. If there is still a standoff, the system will flag the claim as unverified and require further review.

The author also introduces a verification system where records can be signed by two parties, such as the homeowner and an internal reviewer. The signature is bound to a hash of the content it signed, so if the content changes, the signature will no longer be valid. This allows the system to keep a record of disputes and maintain an audit trail.

Overall, the author argues that this approach leads to honest accounting and helps explain how numbers are arrived at. However, it does come with additional costs such as more expensive reads, chattier writes, and a need for a more robust UI to handle disagreements between sources. Nonetheless, the author believes that this approach is worth it in domains where being wrong is expensive and where explanations are required.