Data quality, interoperability, and the future of healthcare AI

In the realm of healthcare artificial intelligence, the reliability of AI systems hinges on the quality, meaning, security, and traceability of the data they process. A record in an electronic health record (EHR) may meet all technical requirements yet still provide misleading clinical information. Similarly, a billing diagnosis might not encapsulate the complete clinical picture, and a laboratory value could use incorrect units or timestamp the entry rather than the clinical event itself.

One prevalent algorithm for assessing population health has been known to underestimate the healthcare needs of Black patients by using healthcare costs as a proxy for healthcare needs.

Interoperability presents another hurdle. The Centre for Medicare & Medicaid Services' interoperability and prior authorization final rule mandates that payers implement or enhance Health Level Seven's Fast Healthcare Interoperability Resources (FHIR) by 2027. FHIR enhances how systems represent and exchange information but cannot rectify subpar source data or align conflicting definitions.

Even when systems successfully exchange values, they may interpret them differently. Consequently, healthcare AI necessitates both syntactic interoperability, which facilitates exchange, and semantic interoperability, which maintains meaning.

Moreover, AI models must undergo testing in their intended environments. Pneumonia detection models trained on chest radiographs from one hospital system often underperform on data from other hospitals due to variations in patient populations, disease prevalence, equipment, workflows, and coding practices. These discrepancies can trigger a phenomenon known as dataset shift.

The FDA's focus on post-market monitoring of AI-enabled medical devices underscores this risk by concentrating on alterations in model inputs, outputs, and real-world performance.

However, deployment is merely the beginning. Continuous operational evidence is critical. For healthcare organizations, trustworthy AI demands five interconnected capabilities: clear data contracts, patient identity and event reconciliation, lineage and provenance, automated quality and drift monitoring, and purpose-based access controls.

The National Institute of Standards and Technology's Artificial Intelligence Risk Management Framework, while not specifically tailored for healthcare, echoes these practices. It emphasizes assessing training-data risks, documenting sources, maintaining provenance, and monitoring systems continuously.

Evaluation criteria should transcend accuracy. They should encompass calibration, predictive value, performance across relevant patient groups, external and temporal validation, and impacts on clinical workflows. In 2024, 79 percent of hospitals utilizing predictive AI reported some level of post-implementation evaluation, but a significant number only assessed part of their models or were uncertain about monitoring practices.

These findings suggest that adoption is outpacing consistent oversight. Therefore, the future of healthcare AI is not only an algorithmic challenge but also an infrastructure concern. Ultimately, a trustworthy AI system in healthcare is as reliant on robust data engineering and vigilant oversight as it is on sophisticated algorithms.