Pennsylvania sues Character.AI over chatbots impersonating medical professionals
Pennsylvania Gov. Josh Shapiro and the Pennsylvania Department of State on Tuesday announced that the agency filed a lawsuit against Character.AI over its artificial intelligence-powered companion chatbots impersonating medical professionals.
The action seeks a preliminary injunction and a court order to stop the company’s chatbots from providing medical advice that only licensed professionals are legally allowed to provide under the state’s Medical Practice Act. It is the first enforcement action from the state agency’s AI Task Force, which was formed in February, tasked with conducting an investigation into AI systems and the unlicensed practice of medicine.
Its investigation concluded that chatbots on Character.AI claimed to be licensed medical professionals, including psychiatrists, available to engage users in conversations about mental health symptoms. The suit alleges that in one instance a chatbot falsely claimed to be a licensed psychiatrist in Pennsylvania and provided a fake Pennsylvania license number. The lawsuit alleges that the company’s platform also allows users to deploy custom AI “characters” that can present themselves as licensed professionals.
“Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” Shapiro said in a press release. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional. My Administration is taking action to protect Pennsylvanians, enforce the law, and make sure new technology is used safely. Pennsylvania will continue leading the way in holding bad actors accountable and setting clear guardrails so people can use new technology responsibly.”
Pennsylvania has also launched a web form on the Department of State’s website where residents can report unlawful behavior or relay complaints about chatbots providing unlicensed health care.
