Stigmatizing patient details can push AI to deprioritize high-risk emergency cases
Artificial intelligence (AI)-driven tools used in emergency triage may be susceptible to the same clinical stigmas associated with human assessments, researchers at UT Southwestern Medical Center reported in a study published in npj Digital Medicine. Their findings suggest this could result in harmful biases and wrongful prioritization of patients.
Researchers from UT Southwestern Medical Center warn that stigmatizing language in patient records could cause AI-driven emergency triage tools to unfairly prioritize or deprioritize patients. The study, published in npj Digital Medicine, examines how AI models process words associated with clinical stigmas, potentially leading to harmful biases.
By comparing neutral and stigmatizing terms, researchers found that details related to mental health, substance use, and patterns of ED use influenced AI decisions. Stigmatizing language, such as "17 visits in the past 12 months" or "active IV drug use," caused higher-priority patients to be deprioritized, even when their immediate medical condition warranted higher attention.
The study highlights the need for AI models to be tested with real patient outcomes before clinical use, as bias varies by model. Age and frequency of ED visits also impacted AI prioritization, although researchers believe these effects stem from the higher medical risks associated with older age rather than age-based bias. The findings emphasize the importance of carefully documenting patient information to ensure fair and accurate AI decision-making in emergency care.
Written by urgent.news from Medical Xpress's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
