As AI enters health care, are we ready?

Artificial intelligence is increasingly being incorporated into how individuals access, interpret and act upon health information. However, the technology is developing at a pace that outstrips the evidence necessary to ascertain whether these systems genuinely assist people in comprehending what's crucial, making well-informed health decisions and understanding what actions to take.

A recent perspective published in the journal Nature Human Behaviour, authored by Rebecca K. Ivic from the University of Alabama, Scott C. Ratzan from CUNY SPH, and Ruth M. Parker from Emory University, introduces the concept of health-literate artificial intelligence and offers four guiding principles for the design and regulation of AI-mediated health communication: comprehension, agency, accountability, and proportionality.

Ivic points out that while we are integrating AI into healthcare faster than we are constructing the necessary evidence to evaluate its effectiveness in helping people understand, decide, and act, AI systems often provide technically plausible responses without adequately conveying the urgency of a situation, the certainty of the information, available alternatives, the need for professional care, or when human judgment should be sought for proper decision-making.

The authors define health-literate AI as systems that align information, guidance, and responsibility with users' abilities, contexts, and needs. This framework shifts the focus from whether AI provides technically accurate or explainable information to whether people can understand what that information means, make informed decisions, and determine the appropriate actions to take.

By combining these elements, the authors aim to assess whether AI-mediated communication is suitable for its intended purpose in health-related settings. Given the ongoing debates and concerns about potential harms associated with AI, there is a need for expert analysis that safeguards individuals from potential risks while harnessing the technology's potential for healthcare.

The authors question whether AI represents the ultimate solution or a complex problem, particularly in medicine, where immediate concerns include whether everyone can comprehend and safely utilize these systems to enhance health outcomes, whether its adoption is cost-effective, and whether it aligns with human values. Ivic proposes a novel model for the current digital information landscape, examining how platforms, institutions, and emerging technologies influence the creation, interpretation, and use of health information.

Instead of solely placing the responsibility on individuals to interpret increasingly intricate automated information, health-literate AI asks how the design and governance of AI systems can facilitate understanding and informed action. The authors also challenge designers to consider how AI could help make health care affordable, a goal that has remained largely unachieved.

The authors argue that the key question is not whether AI can generate an answer but whether people can understand, use, and act on that information appropriately. Health-literate AI should take on more responsibility for making quality health information understandable, actionable, and accountable through a collaborative approach between design and governance to promote responsible AI in healthcare.

The authors emphasize that this is especially critical in high-stakes or uncertain situations. AI systems should clearly differentiate between informing, advising, and deciding, and make these roles evident to users. When automated outputs are insufficient, systems should indicate when human judgment, clinical evaluation, or ethical deliberation is required.

The goal is not to make people better at navigating complex AI but to provide a framework to measure whether the systems themselves are effectively meeting the needs of users. The perspective advocates for the adoption of the health-literate AI principles to guide the design, evaluation, research, and governance of AI systems in health-relevant contexts.

However, the authors acknowledge an important research gap: more studies are needed to determine whether AI genuinely enhances comprehension, informed decision-making, and health-related actions. While AI in healthcare undoubtedly presents opportunities for innovation, it is crucial to ensure that its benefits significantly outweigh the risks through robust research, safeguards, and regulations.

The authors conclude that health-literate AI should enable users to understand health information, make informed decisions, and identify appropriate next steps.