AI tools that take notes for GPs could miss vital information, study suggests

A recent study suggests that artificial intelligence tools designed to take notes during medical consultations may overlook crucial information, potentially leading to medical staff becoming less skilled. Known as ambient AI scribes, these tools are utilized by approximately 40% of General Practitioners (GPs) in the UK. Their function is to listen to conversations between doctors and patients and transcribe the speech into text, producing written notes and letters.

Academics from the University of Edinburgh examined several cases involving these scribes and discovered that while the technology has some advantages, such as allowing clinicians to concentrate on complex tasks, it also presents challenges. Patients may be less forthcoming when they are aware they are being recorded, and doctors might not recognize their own notes.

The study, published in the British Medical Journal's digital health and AI publication, emphasizes the need for further research to understand how these tools operate within specific healthcare settings and how their design supports both patients and clinicians.

The report highlights that AI scribes can assist with administrative work but requires additional research to determine their effectiveness in specific healthcare environments and how their design supports both patients and clinicians. Researchers from the University of Edinburgh noted that while AI tools can capture patients' spoken words, they may fail to capture important aspects of consultations, like facial expressions, gestures, and emotional states.

AI summaries tend to prioritize clinical information over patients' stories and experiences related to their illness. Additionally, patients may be less likely to share sensitive information, such as substance abuse, domestic abuse, or mental health struggles, when they know the consultation is being recorded and processed by AI.

Furthermore, manual note-taking plays a vital role in clinicians' reasoning and reflection on the patient's input. Using AI scribes can result in cognitive offloading for clinicians, meaning the tools take on some of their mental effort. This can benefit clinicians by allowing them to have more meaningful conversations with patients or focus on complex tasks.

However, the study also warns that the tools can reduce memory recall and skill development, and instances have been noted where clinicians fail to recognize their own notes or remember the patient upon subsequent visits.

The researchers argue that in designing these systems, it is essential to preserve the patient's voice rather than overwriting it. They also call for further investigation into the medium and long-term use of these AI tools in specific settings and potential risks, particularly for tools used in countries with healthcare systems different from those where the AI tools were developed.

Dr. Lucas Seuren, from the University of Edinburgh's center for biomedicine, self, and society, commented, "Many clinicians are excited about ambient AI scribes because they promise to cut down on paperwork. However, the experiences of patients are poorly considered, and there are real risks that patients' stories are lost. This can further disadvantage people who already face marginalization in health and social care services."