AI System Faster but Not More Accurate Than Docs for Diagnosing Rheum Disease
(MedPage Today) -- Prof. Valmed, a large language model (LLM) cleared by European regulators for medical diagnosis, outperformed human physicians for diagnosing rheumatologic conditions in terms of speed but not accuracy, a randomized trial showed...
A study published in medRxiv preprint manuscript found that a large language model (LLM) called Prof. Valmed outperformed human doctors in terms of diagnosing rheumatologic conditions, but not in terms of accuracy. The AI system was able to make diagnoses in 94 seconds compared to an average of 206 seconds for human doctors. However, both groups had almost identical accuracy rates, with 33.3% of diagnoses matching the actual published ones.
Prof. Valmed also increased physicians' confidence in their diagnoses, which was higher than the observed accuracy. This suggests that confidence was increased more than correctness, indicating a potential overconfidence issue that needs to be addressed. The study highlights the need to evaluate calibration and behavioral reliance alongside accuracy in AI systems.
Written by urgent.news from MedPage Today's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
