Why RAG on legal text keeps hallucinating dates - and what actually fixed it
A couple of weeks ago I dropped the CRA text (the EU's cybersecurity regulation for IoT devices) into ChatGPT and asked when the main requirements actually kick in. The answer was confident and wrong - it mixed up the date the regulation entered into force (2024) with the date the requirements actually apply (2027). Three years off, stated like an obvious fact. My team (Platanor, embedded…
When I fed the EU's cybersecurity regulation for IoT devices, known as CRA text, into ChatGPT and inquired about when the primary requirements took effect, the AI confidently but incorrectly answered that the regulation entered into force in 2024 and the requirements would apply in 2027. This is a substantial three-year discrepancy, clearly stated with the assurance of a fact.
Our team at Platanor, specializing in embedded security for IoT, has been developing an internal reference on CRA, RED, NIS2, and CSA for several months. Unfortunately, we faced the same kinds of confusion each time we simply fed the regulation PDF into the model. The root of the problem lies not with the AI model, but with how the source material is structured.
Dates are dispersed across various articles without a clear connection between them, token-based chunking often cuts sentences off mid-article, and the model has no way of determining how recent the text is. To address this issue, our team rebuilt the base as a public repository with a proper file structure, rather than relying solely on prompting techniques.
By organizing the content by article headings instead of token boundaries, we ensured that each chunk remained intact, preserving the natural boundaries of each article. We also implemented a source priority system, listing primary source documents first, followed by third-party summaries and our own analysis. This organizational strategy was crucial in preventing the model from misinterpreting the information.
Additionally, we included a verification date on every file, indicating the last time the content had been checked for accuracy (2026-08-10). This date is particularly important for the annexed information, such as the application deadline for Annex I (11 December 2027), which is distinct from the entry-into-force date (10 December 2024).
By incorporating the llms.txt file at the root of the repository, which serves as an index of all files, the model can now select the appropriate data to load without needing to read through the entire repository. Our comprehensive solution, along with the fact-checked base on CRA, RED, NIS2, and CSA, is now available in a single repository.
It can be easily integrated into your own RAG (retrieval-augmented generation) pipeline or installed as a Claude Skill, accessible at https://github.com/Platanor/hardware-compliance-handbook.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — it may contain errors, so check the original before relying on it.
