Healthcare AI: hospitals lack capacity to measure real-world patient outcomes, warns French expert
Emmanuel Bacry warns that laboratory accuracy scores do not establish real-world clinical benefits; says healthcare AI trained largely on Western data may not work elsewhere
In the Bengaluru Tech Summit, Emmanuel Bacry, Scientific Director of France’s Health Data Hub, cautioned that hospitals worldwide struggle to measure the real-world patient outcomes of artificial intelligence (AI) systems. Bacry emphasized that only a few hospitals globally possess the capacity to evaluate medical devices deployed in their systems.
Hospitals face challenges in accessing and integrating patient information to monitor outcomes after discharge, including recovery rates and mortality, making it difficult to ascertain the actual impact of AI-based medical technologies.
Bacry cautioned that hospitals lack capacity to compute the return on investment of AI technologies, which could have significant implications when AI is adopted. In a separate discussion, Bacry highlighted an unintended consequence of AI systems: a doubling of deaths in two out of the six US hospitals where a rules-based AI alert system for ICU staff was deployed in 2022.
Bacry did not identify the study but suggested that the absence of a tested response protocol among hospital personnel might have contributed to the outcome.
Moreover, Bacry pointed to another potential issue: AI systems can identify patterns that even their developers cannot explain, leading to black-box dilemmas. For instance, research has shown that AI models can predict ethnicity from chest X-rays with high accuracy or infer sex from eye images. Bacry also warned about Western bias in healthcare datasets, noting that nearly 70% of global healthcare AI training data comprises male, white, and Western populations.
He advocated for smaller, locally tailored AI models trained on representative population data to avoid potential disastrous consequences, especially in developing countries.
Rewritten by urgent.news; may contain errors — check the original.
Source: thehindu.com
This story
This is one outlet's version. Read the fullest account.