New study reveals AI chatbots recognize unnecessary medical tests, but stumble where human doctors disagree
In an era where patients and clinicians increasingly turn to artificial intelligence for medical advice, a critical question emerges: Does AI know when not to test or treat? A new joint study from Ben-Gurion University of the Negev (BGU) and Clalit Health Services reveals that leading large language models (LLMs) successfully identify when medical interventions are redundant in the vast majority…
A new study from Ben-Gurion University of the Negev and Clalit Health Services in Israel has shed light on the capabilities of artificial intelligence in the medical field. The researchers found that leading language models (LLMs) generally recognize when medical tests or treatments are unnecessary, but struggle in situations where human doctors themselves are uncertain.
The study, published in Annals of Internal Medicine, comes as patients and clinicians turn to AI for medical advice. While extra tests can provide comfort to patients, they often lead to unnecessary worry, additional procedures, and potentially harmful consequences. Conversely, being too conservative with diagnostics can result in missed opportunities for crucial interventions.
The researchers evaluated six top LLMs across hundreds of clinical scenarios derived from the Choosing Wisely campaign, which aims to eliminate low-value care practices. The results showed that AI models overtested in approximately 5.4% of cases, while failing to recommend necessary care in only 0.6% of instances. The majority of errors were related to practices like ECGs and thyroid ultrasounds in ambiguous or conflicting cases.
Despite their ability to know when to stop testing, the AI models still struggled in gray areas where human doctors themselves hesitate. The study highlights the delicate balance AI must strike between unnecessary overtesting and overly conservative diagnostics, especially when dealing with anxious patients.
Written by urgent.news from Medical Xpress's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
