AI Could Push High-Risk ER Patients Down the Queue, Study Warns
Researchers found AI emergency triage decisions could be influenced by details including frequent ER use and psychiatric history.
A recent study warns that artificial intelligence systems used to prioritize Emergency Department (ER) patients may unintentionally deprioritize those at higher risk based on certain information in their medical records. Researchers from two academic emergency departments examined how AI models from three major AI families—Gemma by Google, Qwen by Alibaba, and DeepSeek by Chinese AI company DeepSeek—reacted to potentially stigmatizing details such as frequent emergency visits, psychiatric emergencies, and substance-use histories.
The study found that including such information could shift AI models' assessments away from patients who were actually at greater risk of deterioration. This means that aspects of a patient's history, unrelated to the immediate severity of their condition, could influence the AI's prioritization decisions, potentially disadvantaging higher-risk individuals.
The findings suggest that while AI can assist in processing medical data, it is crucial to test for bias before relying on these systems for critical clinical decisions, especially in cases where patients may already face stigma due to their medical histories.
Written by urgent.news from Newsweek's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
