How I stopped an LLM from fabricating citations (a coin-museum story)
I started collecting ancient and world coins this year. It got out of hand fast, the way these things do. What I wanted next was a catalogue that read like a story instead of a spreadsheet, and that catalogue did not exist, so I built it. Numistoria is my collection presented as a museum: twenty themed "worlds" that read as one continuous story, with an exhibit page for every coin, whether I own…
In this story, the author recounts their journey of creating a museum of ancient and world coins, called Numistoria. They built a unique system that allows an LLM to write engaging historical narratives about the coins while strictly preventing the model from fabricating any factual information about the coins themselves. The system is divided into two main parts: Coinex, which handles private data, and Numistoria, which presents the public-facing content.
Coinex tracks the coins' origin, sale information, and future acquisitions, while Numistoria generates the connected narrative for each coin using the Claude LLM. The model receives a curated list of verified sources for each coin and can only select from these pre-approved citations, ensuring that no false or fabricated references appear in the final output.
Additionally, a reviewer flag is added to the deep dive to indicate that the information is machine-generated, further emphasizing the AI's involvement. The author also describes a hunt system where a curator model proposes coin acquisitions within a budget, and a weekly GitHub Actions job searches for and verifies potential purchases from reputable dealers.
The resulting articles are then sent to the user via email, highlighting any white whale acquisitions that exceed the budget. The entire system is designed to be simple and efficient, with a focus on human oversight and control over the AI's output. The site is currently live at numistoria.danmat.dev and the open-source code can be found at github.com/DanMat/Numistoria.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
