The benefits of medical AI assistance vary based on user expertise
Study finds non-experts deferred to LLM-based diagnostic assistance, even when it was wrong, while clinicians caught AI errors.
A recent study by researchers at MIT and elsewhere has found that the benefits of medical AI assistance vary based on the user's expertise. While AI assistance generally improved the accuracy of both non-experts and clinicians in diagnosing skin diseases, the impact of AI explainability methods differed depending on the users' knowledge level.
For non-experts, AI explainability methods led to improved diagnostic accuracy due to deference to the AI system, with users trusting LLM-based explanations regardless of their correctness and finding vague explanations more convincing. In contrast, clinicians performed best when given only the model's prediction without accompanying explanations.
The study's authors emphasize the importance of designing AI systems with users in mind and developing explainability methods that encourage critical thinking rather than overreliance on the model, as those with the least medical knowledge are most likely to be misled by erroneous AI outputs.
Brief written by urgent.news from MIT News AI's own syndicated text. Machine-written — may contain errors; check the original before relying on it.
