The FDA’s new draft guidance on Bayesian methods in clinical trials has been hailed by some as a breakthrough that could speed drug development. But statisticians and researchers are divided on whether it represents a genuine shift, formalizes what was already happening, or loosens important safeguards.

The guidance, released on January 12, allows Bayesian statistical methods to be used as the primary analysis in pivotal trials for drugs and biologics — something that was previously standard only for medical devices.

“The main excitement from the FDA draft guidance is allowing the Bayesian methodology to be used in clinical trials, including pivotal studies as the primary analysis. To me, it’s a breakthrough,” said Yuan Ji, PhD, professor of biostatistics at The University of Chicago, Chicago, who has written about the guidance.

FDA Commissioner Marty Makary called Bayesian methods “a leap forward” that could address “two of the biggest problems of drug development: high costs and long timelines.”

But not everyone shares the enthusiasm.

“I think it’s less of a revolution and less useful than many people suppose,” said Stephen Senn, PhD, a 30-year consultant statistician for the pharmaceutical industry who has worked extensively on clinical trial methodology. “I’m not averse to it, genuinely. I just don’t like hype.”

What’s Different?

Traditional clinical trials use frequentist statistics, which treat each trial as a standalone evaluation. The key question is, if this drug had no effect, how likely would we be to see results this strong by chance? If that probability (the P value) falls below a threshold — typically 0.05 — the drug is deemed effective.

Bayesian methods ask a different question: Given everything we knew before and the new trial data, what’s the probability the drug works?

“I think the Bayesian methods are very natural for human logic,” Ji said. “It’s concluding or making decisions based on the probability of what’s being considered as a central question, like if the drug is better than the existing standard of care, and one can make that decision based on a probabilistic statement rather than P values.”

The practical appeal is that Bayesian methods can formally incorporate prior information — from earlier trials, related diseases, or adult studies when evaluating pediatric populations. “It’s potentially going to lead to faster and more efficient clinical trials,” Ji said.

In an email response to Medscape Medical News’s request for comment, the US Department of Health and Human Services (HHS) spokesperson said the guidance signals broader openness to Bayesian approaches. “Sponsors should interpret this guidance as a signal that we are more open to Bayesian statistics generally,” the spokesperson said. “Pediatric studies and rare diseases are situations where there is a higher need and openness for regulatory flexibility, but the scientific rationale that underlies these methods can be justifiable in other situations.”

The Skeptics’ Concern: Subjectivity

Bayesian methods require researchers to specify a “prior distribution” — essentially, a mathematical statement of what they believed before the trial began. Critics worry this introduces subjectivity into what should be an objective process.

“The worrisome part with the Bayesians has always been the subjectivity of priors that only reflect beliefs,” said Deborah Mayo, PhD, professor emerita of philosophy at Virginia Tech, Virginia, who has written extensively on statistical evidence. “Nobody has ever been able to explain how to connect beliefs and probability in a way that we can all agree on.”

Mayo’s concern focuses on a provision in the guidance that allows certain Bayesian designs — particularly those using “informative priors” that borrow from external data — to forgo traditional Type I error control. Type I error probability is the probability of inferring an effect, like concluding a drug works when it doesn’t.

“If somebody says, ‘Look, I really believe this drug works,’ then it might be thought they don’t have to show their method had a low probability of finding evidence that it seemed to work, even if that’s wrong,” Mayo said. “It suggests that it would be to suspend the regulations and safeguards that the FDA has long had in place.”

She pointed to the case of Scott Harkonen, the InterMune CEO convicted of wire fraud in 2009 for issuing a misleading press release about phase 3 results. With new standards, she wonders whether researchers could still be held accountable for doing something like this. “Think of how much easier it would be for somebody to say, ‘Well, we really believed it,’ or ‘Given my informative prior, this seemed plausible,’” she said.

She also noted, however, “There are plenty of welcome safeguards in this document.”

Will a Middle Ground Be Good Enough?

“When it comes to applying a technology or new method, there’s always a risk of abusing it,” Ji said. “The abuse of a technology is not the fault of the technology — it’s the people who abuse it. So I think this is where regulation is needed.”

“The regulators need to be fully equipped with the Bayesian knowledge to be able to see if there’s any under-the-table manipulation or abusive usage of prior distribution that could lead to favorable results that are not necessarily factual,” Ji added.

Andrew Gelman, PhD, Higgins professor of statistics at Columbia University, New York City, and lead author of the textbook “Bayesian Data Analysis,” is skeptical of the traditional frequentist framework — particularly its focus on Type I error, which measures the risk of approving a drug that has no effect at all.

“I think that it’s rare that the true effect is zero,” Gelman said. “Usually, things that are being studied have effects. The effects can vary.”

Gelman argued for weighing evidence against consequences. “It can be okay to say you think a treatment is effective, even if you’re not sure,” he said. “Sometimes you have partial information, and if there’s something that seems positive but you’re not sure, then you should be looking at the costs and the benefits.”

Per the HHS response to Medscape Medical News: “One common misconception is that sponsors always need to calibrate to a particular Type I error rate in every circumstance. Another misconception is that the use of Bayesian methods always means an easier path to approval. Carefully constructed prior distributions should be based on all relevant available information, including both potentially positive and negative information.”

Will the Industry Actually Change?

The deeper question may be whether drug companies will embrace Bayesian methods — or stick with what they know.

Witold Więcek, PhD, a biostatistician and consulting director of the Development Innovation Lab at the University of Chicago, has written critically about the guidance. Drug developers “crave specificity” from regulators, he said. “If you create guidance which says that it’s good to do X, but we can’t tell you specifically how we see you doing X, then drug makers will see it as, ‘Well, our uncertainty about doing X is exactly where it was before,’ and it’s not really an invitation to doing more Bayesian clinical trials.”

He also pointed to structural barriers. “For this type of trial, it’s not going to be a pharmaceutical company itself doing all of the legwork, but the CRO (contract research organization) which they are working with, and CROs are extremely risk-averse in the first place. It’s kind of hard to see whether guidance like this will immediately lead to a big shift in how people do trials.”

There are also practical questions about whether Bayesian methods can be implemented clearly and reliably. Senn, who has served on data safety monitoring boards using Bayesian methods, raised practical concerns about interpretability. “I’ve sat on data safety monitoring boards where the Bayesian Time Machine, which is one of the papers which is cited [in the guidance], has been used, and I found it very difficult to find out exactly what’s going on.”

Senn noted that borrowing historical data isn’t straightforward. “If you have a look at the development of HIV therapy, you will find, even during the time that the trials were being run, later cohorts have better survival than earlier ones,” he said, due to improvements in managing opportunistic infections. “If you believe in medical progress, then it’s more or less obvious that historical controls are not the same as concurrent ones.”

The Bottom Line

The guidance may prove most consequential in areas where traditional trials are impractical — rare diseases, pediatric indications, and situations where patient populations are too small for conventional approaches.

“The basic idea behind randomized clinical trials is that you study things now under ideal conditions, because you will gain information that will enable you to treat many, many future patients,” Senn said. “But if the future patients are very few, then that logic no longer applies, like in a rare disease. So you may say, ‘Well, I actually want to do something now, as soon as possible,’ and the Bayesian approach is one way to deal with this.”

Whether the guidance transforms drug development more broadly — or remains a niche tool — may depend less on statistics than on institutional culture.

“I’m very excited about this new guidance,” Ji said. “I think it finally opens the possibility of using Bayesian methods for these pivotal studies in drugs and biologics. And I think the focus in the near future will shift from whether or not it’s allowed to how to implement it in a responsible way.”