Public Citizen Urges HHS to Create Guardrails for Use of AI in Health Care
Washington, D.C. – Public Citizen today released its recommendations to the U.S. Department of Health and Human Services (HHS) on the use of artificial intelligence (AI) in health care, noting that consistent human oversight is needed to limit potential harms, protect patients, ensure accountability, and provide equitable access to care. The recommendations were provided in response to the Biden Administration’s Executive Action on Artificial Intelligence, which Public Citizen has called a vital step to begin the long process of regulating rapidly advancing AI technology.
The use in AI in health care has already produced deeply concerning results. In 2020, researchers found that a medical chatbot urged a fake patient to kill themselves. This is far from a hypothetical scenario; a mandied by suicide after speaking with a chatbot over several weeks in 2023 in a conversation that reportedly turned “increasingly confusing and harmful”. What’s more, algorithm-driven payment denials are cutting Medicare Advantage patients off from the care they need, and AI embedded in patient records is prompting doctors to recommend treatments. Meanwhile, some companies are announcing plans to replace nurseswith AI technology.
“We need regulators to provide clear-eyed assessments of the risks of the use of AI,” says Eagan Kemp, Health Care Policy Advocate at Public Citizen. “Human review and oversight of AI and its implications will be essential as the technology expands in health care. We need strong data protections, mechanisms for monitoring adverse events, and efforts to mitigate the ongoing risks of such events.”
In its comments to HHS, Public Citizen said that rulemaking should ensure transparency when health care decisions are made by AI, noting the patient and their physician should have the right to an understandable explanation of the decision, the right to request human review, and the right to have the decision appealed to a human. Public Citizen also called for HHS to include protections required for sharing, storing, and anonymizing data for AI-enabled devices; enforcement efforts and penalties for failure to meet relevant standards; and mechanisms to protect patient data.
The group also noted that strong equity principles must underlie the deployment of the technology, and companies and providers must be given clear guidance on how equity principles are to be incorporated into their AI-enabled technologies.
Additionally, Public Citizen says AI and AI-enabled tools used in drug development must help meet public health needs, including in rare and traditionally neglected disease areas, rather than only benefit private interests and pharmaceutical company profits and patients with the greatest financial means and disease areas with the most available data.
“AI must be monitored to help protect patients, providers, and the public as they access needed health care. It should not be used as a tool to increase already enormous corporate profits,” says Kemp. “Regulators have a lot of work to implement guardrails and ensure adequate oversight of this technology.”
“AI companies are rushing to infuse AI tools into the health care system and are directly marketing products to consumers,” added Kemp. “The risks are enormous – from bad therapeutic advice to mistaken diagnoses and more – and the marketing is outpacing the medical evidence and needed regulatory protections. It’s crucial that HHS get strong rules in place to protect patients and hold AI and health care companies accountable when things go wrong.”
Read Public Citizen’s fact sheet on AI in health care.