A Utah pilot program that will allow an AI-powered system to authorize prescription refills for select psychiatric medications is raising patient safety concerns and doubts about its compliance with federal statutes.

The state’s Department of Commerce entered into a 1-year agreement with the telehealth company Legion Health in March. Utah said it is waiving professional licensing requirements and will forego enforcement during the pilot, so long as the company adheres to state rules for telehealth providers.

In its proposal, Legion Health said its AI refill system could help alleviate behavioral health provider shortages and that it will allow patients to “get high-quality mental healthcare much more quickly and much more affordably than ever before.”

Echoing Legion’s language, the state said on a landing page that routine prescription renewals “are high volume and high friction,” and that automation through the AI platform will create faster access and better care.

John Torous, MD, director of the Digital Psychiatry Division at Beth Israel Deaconess Medical Center in Boston, said he’s concerned that the program is being tested on state residents.

“It seems like no one has done even the basic research on this,” he told Medscape Medical News. “Do patients want it? Do clinicians want it? Will it be useful? Where does it work? Where does it not work? What is it good at? What is it bad at?”

Torous, associate professor of psychiatry at Harvard Medical School in Boston, said he’s not against assessing whether an AI-powered refill system is beneficial but believes there should be more transparency and more safety plans before rolling it out to the public.

The American Psychiatric Association (APA) said in a statement that while the technology could potentially increase access, “prescribing psychiatric medication is the practice of medicine and must remain under the care of a licensed physician.” AI systems that issue or renew prescriptions “without continuous physician involvement raise serious concerns about patient safety, quality of care, and accountability,” the APA noted.

Neither Legion Health nor Utah Department of Commerce officials responded to requests for comment.

Safety Concerns

There are no precise details in the proposal as to how the AI refill system would work. It appears that any Utah resident could sign up for Legion Health’s service for a $20 monthly fee and then submit a refill request through the company’s app. Patients must submit photographic evidence of the current medication and have no dosing changes. It’s unclear whether there is an additional cost for refills.

According to the company, an AI-guided chatbot asks patients about their medication’s efficacy, side effects, and any safety concerns. It then screens for signs of suicidality, mania, or hypomania — any of which would disqualify them from receiving a refill.

Under the company’s agreement with the state, the first 250 refill requests must be reviewed by a licensed physician before a prescription is sent to a pharmacy. The following 1000 requests undergo retrospective review by a clinician, and thereafter the AI system is monitored on a monthly basis.

Refills are limited to the formulary posted in the agreement, which includes selective serotonin reuptake inhibitors and other medications for anxiety, depression, and insomnia. The program will not refill benzodiazepines, antipsychotics, or other controlled substances.

Since late 2025, Utah has partnered with the telehealth platform Doctronic to issue AI-generated prescriptions for chronic conditions. It uses a chatbot —overseen by physicians — to assess refill requests; controlled substances are excluded. However, an independent security review by Mindgard — a cybersecurity research firm that specializes in stress testing AI-driven healthcare systems — revealed vulnerabilities in the Doctronic chatbot used for telehealth, not the tool used for prescription writing. 

When researchers gave the AI prescribing system adversarial prompts, it recommended methamphetamine for social withdrawal and more than tripled a patient’s oxycodone dose, among other misfires.

Torous said that because any AI program can be manipulated, it’s important to know more about Legion’s system. Legion has not publicly provided performance or safety data.

The system is designed to keep refilling medications — a maximum of 10 between provider reviews or 6 months, whichever is sooner. That seems robotic, said Torous.

“Sometimes we want to help people get off medications,” he said. A system designed to maximize refills “doesn’t reflect how psychiatry works in a modern era,” he added.

The field continues to grapple with AI-assisted therapy and treatment. In an APA member survey, 40% of psychiatrists viewed it as riskier than traditional approaches, 38% believed it would boost clinicians’ effectiveness, and 23% were uncertain.

The Legion AI platform should prompt the community to educate themselves about such technology, said Torous. Without that knowledge, it will be harder to “move the field where we want to go,” he said.

Violation of FDA Law?

While Legion’s system is being protected from state enforcement, it may not be free from federal oversight, said Daniel Aaron, MD, JD, associate professor at the S.J. Quinney College of Law at the University of Utah in Salt Lake City.

In an April 13 JAMA Viewpoint on the Doctronic prescription program, Aaron and Christopher Robertson, JD, PhD, of the Boston University School of Law in Boston, contend that the system is likely illegal because AI is not a licensed practitioner, as required by FDA statutes.

“Without the licensed practitioner’s prescription, drugs become misbranded, making their sale a crime,” they wrote.

Aaron told Medscape Medical News that the Legion Health AI prescribing program raises the same questions, as it also does not use a licensed practitioner to write prescriptions.

He added that there is no evidence that either Doctronic or Legion Health discussed their technologies with the FDA. However, he also noted that the Trump administration has promoted AI and encouraged deregulation of the technology.

Still, the greenlighting of these AI prescribing systems in Utah is outside the regulatory norms, said Aaron.

“This product should have been tested beforehand through clinical trials, which is the way that all other devices are supposed to come to market,” he said. “This is a huge deviation from FDA’s normal standards, and what is expected in the US marketplace for new medical devices,” he said.

Aaron emphasized that devices and medications require solid safety and efficacy data to protect patients from harm. He also warned that charging for the refill service could leave patients worse off both health-wise and financially.

Torous reported having no relevant financial relationships. Aaron reported that he receives grant support from Arnold Ventures to study the FDA.

Alicia Ault is a St. Petersburg, Florida-based freelance journalist whose work has appeared in many health and science publications, including Smithsonian.com. You can find her on X @aliciaault and on Bluesky @aliciaault.bsky.social.