Healthcare AI Adoption Spurs Patient Confusion, Study Finds

A June 2024 study shows that most patients do not understand how AI is used in their healthcare, with 62% reporting confusion or uncertainty about AI’s influence on diagnosis and treatment decisions, underscoring the need for transparent communication and robust governance frameworks HealthTech News Journal of Medical Ethics.

A peer-reviewed study published this week in the Journal of Medical Ethics surveyed over 1,200 patients across major U.S. health systems and found that 62% were unclear about how AI tools contributed to their care, while 47% expressed concerns about the lack of explanation from providers regarding AI-driven decisions Journal of Medical Ethics. The study highlights a growing disconnect as AI adoption accelerates in clinical workflows, with patients reporting confusion about AI’s role in diagnosis, treatment recommendations, and data privacy HealthTech News.

This confusion poses significant risks for regulated healthcare enterprises, especially as the EU AI Act and U.S. frameworks like HIPAA and the FDA’s Good Machine Learning Practice (GMLP) emphasize transparency, explainability, and informed consent in AI-enabled care EU AI Act. Failure to clearly communicate AI’s use and limitations may not only erode patient trust but also increase exposure to regulatory scrutiny and legal liability. The study’s findings reinforce the need for health systems to align patient communication with regulatory requirements for transparency and documentation Journal of Medical Ethics.

CTOs, CISOs, and Compliance Officers should immediately review their organization’s patient communication protocols regarding AI-enabled tools. In the next 30-90 days, leaders must assess whether current disclosures meet the transparency and informed consent standards set by the EU AI Act, HIPAA, and FDA guidance. Enterprises should prioritize staff training on AI explainability and update governance frameworks to ensure patients receive clear, accurate information about AI’s role in their care HealthTech News.

What This Means for Enterprise AI

Healthcare enterprises deploying AI must ensure that patient-facing staff can clearly explain when and how AI is used in diagnosis or treatment, as required by the EU AI Act’s transparency provisions and HIPAA’s informed consent mandates EU AI Act. Failure to do so increases the risk of regulatory penalties and reputational damage if patients perceive AI as a “black box” or feel misled about their care Journal of Medical Ethics.

Operationally, this means updating patient intake and consent forms to explicitly reference AI-enabled tools, providing plain-language educational materials, and training clinicians to answer patient questions about AI’s role in care decisions. Compliance teams should audit current workflows for gaps in AI transparency and document all patient communications as part of ongoing risk management HealthTech News.

In the next quarter, expect increased scrutiny from regulators and accrediting bodies on how AI is disclosed to patients. Proactive alignment with the latest EU and U.S. guidance on AI explainability will be critical for maintaining trust and avoiding compliance pitfalls.