The AI is listening: Implications of machine-generated clinical notes on health care
Artificial intelligence tools are being touted as a solution for so-called pajama time—the time doctors inevitably spend compiling patient notes after working hours. As hospitals increasingly deploy AI scribes to listen and take notes during patient encounters, the data the scribes produce becomes the scaffold on which insights about health care, from billing to research to clinical…
The rise of artificial intelligence tools for generating clinical notes has sparked a debate about the implications for healthcare. As hospitals adopt AI scribes to transcribe patient encounters during work hours, the data produced by these systems becomes foundational for insights into billing, research, and clinical decision-making. However, the reliance on machine-generated notes raises concerns about data integrity and potential inaccuracies.
According to Dr. Taofeeq Oluwatosin Togunwa and Dr. Jodyn Platt from the University of Michigan Medical School, AI scribes are increasingly becoming the primary source of data in the healthcare pipeline. Unlike human clinicians who traditionally filtered information, machines now decide what becomes data. This shift presents challenges, particularly when AI systems are proprietary with opaque training data and may not be optimized for diverse patient demographics.
One major concern is the performance of AI systems with patients who have different accents or linguistic backgrounds. Platt noted that these models, often trained on English and American accents, may perform poorly with patients who have non-standard speech patterns. The resulting "hallucinations" – fabrications introduced by AI when inputs deviate from training data – can become part of the official patient records.
While clinicians are expected to review these notes for accuracy, the process is often lacking due to the convenience of general note quality. Additionally, AI scribes can become less reliable over time if not regularly updated, leading to performance drift when real-life data diverges from training data.
To combat these issues, the authors advocate for continuous auditing of AI scribe systems to monitor performance drift and ensure data integrity. They also emphasize the importance of measurement beyond routine metrics, focusing on critical outcomes such as patient diagnosis speed, clinician-patient interaction efficiency, and equitable access to healthcare benefits. By balancing time savings with patient outcomes, health systems can maximize the positive impact of AI scribes while mitigating potential risks.
Written by urgent.news from Medical Xpress's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
