A score meant to help primary care clinicians in prescribing opioids missed more than 90% of cases in which patients went on to overdose or be diagnosed with opioid use disorder, according to a recent study published in the Journal of General Internal Medicine.

The model, developed by electronic health records company Epic Systems, was built on data from nearly half a million patients across three health systems.

The Opioid Risk Score (ORS) from Epic is “not ready for prime time,” said Scott G. Weiner, MD, MPH, associate professor of emergency medicine at Harvard Medical School in Boston, after reviewing the new paper. Weiner, who researches safe opioid prescribing, was not involved in the study.

Researchers and clinicians have sought for many years to find a way to use clinical decision support tools to better identify patients for whom an opioid would put them at risk for adverse outcomes.

But research has shown external validation of opioid risk models is rare, and what data exist do not provide insight into how these tools work in clinical practice. In an unrelated matter, the US FDA is reviewing a citizen’s petition seeking more oversight of a similar but unrelated risk score called NarxCare.

Questions About Screening Tool

Stephanie Hooker, PhD, MPH, research investigator at the nonprofit HealthPartners Institute in Minneapolis, said she and her colleagues conducted the study because of questions that arose during their research of another support tool called Opioid Wizard, which was developed by their institution.

In that analysis, some clinics used the ORS to attempt to identify patients with a high risk for problems with opioids. Hooker and her team heard from clinicians that the tool made it appear few patients had substance use concerns, which seemed doubtful, Hooker said.

“They were getting feedback from the nurses and clinicians using the risk score that everyone was screening negative for substance use concerns,” Hooker said.

So Hooker and her colleagues designed a substudy to evaluate ORS, using claims and electronic records from over 700,000 patients to gauge the accuracy of the tool to predict new diagnoses of opioid use disorder or overdoses within 1 year of score calculation. Patients did not have a diagnosis of opioid use disorder at baseline and had a primary care encounter over a nearly 2-year period starting in April 2021.

Results showed a false-negative rate of 92.2%, where patients who already had a diagnosis of opioid use disorder or a previous overdose were classified as low risk.

Diagnoses of opioid use disorder and opioid overdoses were rare in the 12 months following index visits (n = 2362; 0.3%). The Epic model showed low sensitivity to correctly identify people at risk (0.0783; 95% CI, 0.0675-0.0892). But it was almost entirely able to correctly identify patients not at risk (specificity, 0.9965; 95% CI, 0.9963-0.9966).

The results indicate that clinicians would miss many chances to counsel patients at risk for opioid use disorder and overdose and offer them medications such as buprenorphine, which is still underused in primary care, Hooker said.

Clinicians need more information about the effects — both intended and unintended — of these scores on medical practice.

“Would a high score make it less likely for a surgeon to prescribe an appropriate short course of opioids after a procedure?” Weiner said. “Would a patient who has no opioid use disorder be treated with stigma by staff because the score was high?”

Epic told Medscape Medical News in an email that it “appreciates the feedback shared” in the current study.

The study was supported by the National Institute on Drug Abuse’s Clinical Trials Network. Hooker disclosed receiving a Loan Repayment Award from the National Institute on Drug Abuse for her work on this project. Other co-authors reported having no relevant financial disclosures. Weiner reported receiving personal fees from Vertex Pharmaceuticals and grants from the National Institute on Drug Abuse, among others.

Kerry Dooley Young is a freelance journalist based in Washington, DC. She has covered medical research and healthcare policy for more than 20 years.