Utah has launched an autonomous prescribing pilot program, partnering with a physician-led artificial intelligence (AI) company to become the “first state-approved program in the country that allows an AI system to legally participate in medical decision-making for prescription renewals,” according to the Utah Department of Commerce.

The program, powered by the AI platform Doctronic, will approve a limited set of medication renewals for chronic conditions such as diabetes and hypertension.

The announcement comes as physician groups and legislators debate the role of AI in medicine, even as payers and health systems move quickly to adopt the technology to cut costs, reduce care gaps, and ease administrative burden.

Several states, including Utah, have moved to place guardrails on AI in sensitive settings and banned AI systems from presenting themselves as clinicians.

Doctronic was co-founded by Adam Oskowitz, MD, a vascular surgeon and associate professor of surgery at the University of California, San Francisco. The company did not respond to a request for comment about the pilot program.

‘Doctor, Not Device’

Under the pilot, patients seeking refills complete an online intake form. Patients must confirm they are in Utah, then enter the medication they want refilled, and select an in-state pharmacy for fulfillment. Users must also upload an ID, take a selfie, and pay a $4 service fee, according to the renewal website.

The AI system reviews the information before issuing the refill, but the website does not detail what clinical factors are evaluated. However, in a recent interview with The Salt Lake Tribune, Oskowitz said the AI can verify if the user has been previously prescribed the requested drug by reviewing a shared pharmacy database. He also said that uncertain cases are referred to a Utah-licensed human physician.

Utah’s Office of Artificial Intelligence Policy (OAIP) approved the program as part of its regulatory sandbox, which allows companies and state agencies to test and evaluate emerging AI technologies under state supervision.

State leaders have emphasized that the program is intended to support clinicians and to offer patients a cost-effective way to access routine prescriptions.

The partnership “reinforces the principle of ‘doctor, not device,’” Kirk Cullimore, Utah senator who sponsored the 2024 law that created the regulatory sandbox framework, said in a statement.

The pilot will run for 12 months and roll out in two phases, an OAIP official told Medscape Medical News. Phase 1 will include 250 patients, with prescription renewals reviewed by a physician before they’re issued. Phase 2 will expand the program to 1000 patients and introduce retrospective review for prescription renewals.

While AI-assisted refills could reduce physicians’ administrative burden and improve medication adherence, the OAIP official said the state has no plans at this time to expand the service beyond the pilot.

From Symptom Checker to Prescription Renewals

Doctronic entered the healthcare market in 2023 with an AI-powered symptom-checking platform branded as “your private and personal AI doctor.” The company website says the tool is a fast, free, and anonymous way for users to ask health questions and determine when to seek further care.

According to the company’s website, the system has provided more than 21 million consultations, and users can choose to pay for a video visit with a licensed physician after the free AI-generated chat. The Utah prescription pilot builds on that technology.

In a preprint posted to medRxiv, the company’s research indicates that its AI system’s top diagnosis matched a clinician’s assessment in 81% of cases and that its treatment plans aligned 99.2% of the time. The study, authored by Doctronic Research Group, has not been peer-reviewed.

Doctronic’s expansion into autonomous prescribing occurs alongside increasing interest and investment in healthcare AI. Lightspeed Venture Partners, a Doctronic investor, also backs Abridge, a physician-led AI scribe vendor used by several large health systems.

Still, for all the momentum behind clinical AI, relying on algorithms for medical care raises concerns about safety and oversight.

“Access to refills is definitely a real barrier,” Tinglong Dai, PhD, professor at the Johns Hopkins Carey Business School in Baltimore, and co-chair of the Johns Hopkins Workgroup on AI and Healthcare, told Medscape Medical News.

If Utah’s program works as described, with “clear rules, hard exclusions, and a human escape hatch,” the upside is promising — fewer lapses in therapy and less time clinicians must spend on routine tasks, Dai said.

What’s riskier, Dai said, is treating narrow clinical tasks as inherently safe for automation without rigorous regulation and continuous monitoring. In a recent npj Digital Medicine study, he and his co-authors found that AI systems can appear reliable while missing clinically meaningful errors.

Autonomous systems “are not close to specialist performance and definitely not where you would want it if you treated it like a real medical device,” he said.

His biggest concern is that small failures, such as an outdated medication list, a missed allergy, or overdue monitoring labs, could accumulate, possibly harming patients who should see their physicians before renewing their prescriptions.

“The obvious solution is to keep the scope tight, make escalation easy, and audit it properly,” said Dai. “If we’re not measuring misses and near-misses, it’ll drift by design.”

Steph Weber is an award-winning freelance journalist specializing in healthcare and law.