AI agents are checking the scientific literature — and spotting decades-old errors

For years, chemists have relied on handbook values for a molecule's boiling point to identify substances and plan processes like distillation. However, an artificial intelligence model has uncovered that some trusted numbers in a reference database have been incorrect for decades. Sebastian Pios, a theoretical chemist at Zhejiang Lab in Hangzhou, China, was utilizing an AI system to predict boiling points of various molecules when it started generating values that did not align with long-established entries in a 75-year-old reference database.

Initially, Pios believed the model was at fault. Upon manually scrutinizing the original literature, he discovered that the reference data were indeed erroneous, not the AI model. Pios found two other instances where his AI model had identified errors in older papers and reference books - a typo in one case and incorrect values for centuries-old boiling-point measurements in another.

Both mistakes would likely have caused considerable trouble for researchers who had used the database. Pios notes that many scientists are now turning to AI as a tool for auditing scientific knowledge, including checking databases and hunting for errors in published papers and conference proceedings. While AI fact-checking tools remain unreliable arbiters of the scientific corpus, they still offer significant advantages in terms of speed and efficiency compared to manual scrutiny, according to computer scientist Odd Erik Gundersen of the Norwegian University of Science and Technology in Trondheim.