AI is already deciding whether or not people receive medical care

Recent weeks have seen a surge of concern over the potential risks AI poses to humanity, such as the threat of human extinction. However, the reality is that AI programs are already having a direct and harmful impact on people's physical health. UnitedHealth Group, the largest health insurer in the US, has been accused of using AI to deny payments to patients, despite doctors recommending necessary care, and is facing a class-action lawsuit.

Humana and Cigna are also under investigation for similar allegations. Meanwhile, Medicare has introduced an AI tool to aid in determining whether to approve prior authorizations, and doctors are receiving reports from patients who are distraught over not being able to obtain medicine for managing chronic pain.

The legal complaints in these cases paint a harrowing picture: families desperately seeking to keep an elderly loved one who has experienced a serious fall in a nursing home, while doctors find themselves frustrated when an algorithm cuts off a patient requiring months of rehabilitation after just a few weeks. This represents the current threat AI poses to people's health.

While the conversation may be focused on more Hollywood-inspired concerns, private insurers and the Trump administration have already begun deploying AI programs to scrutinize health insurance claims, with evidence suggesting these programs may not always function optimally and could put people's health at risk.

Until legal proceedings are resolved or policymakers devise a method to regulate how insurers utilize this new technology, it falls upon individuals to ensure that an AI program does not strip them of insurance benefits they are entitled to. Mika Hamer, a health services researcher at the University of Maryland, explains that clinicians' decisions are heavily influenced by a patient's unique circumstances, which often cannot be accurately quantified and thus are not suitable for inclusion in models.

Relying solely on models may cause us to lose the human touch in healthcare. Patients report that private health insurers are leveraging AI to deny claims. Insurers and patients have always engaged in disputes over medical claims, with policyholders seeking maximum coverage and payers striving to pay the least possible. Health insurers have long employed algorithms, using factors such as age, weight, and smoking habits to estimate medical care needs.

However, the newer AI programs can make precise predictions about future health based on the input data, which are then used in approving or denying medical claims. The exact information used to generate these predictions and who bears ultimate authority over them — the AI or the human — is frequently unclear to the public. Daniel Schwarcz, a law professor at the University of Minnesota, emphasizes that human review is crucial to ensure meaningful substantive judgment, while cautioning that distinguishing between AI supporting human processing and AI replacing humans with passive defaulting is extremely difficult, particularly in the realm of regulations.