Science and medicine are human endeavors and, therefore, are not immune to the imperfections of those who practice them. Although authority is often associated with these disciplines, it remains subject to human error. Since the end of the last century, growing evidence has shown that scientific findings are not always supported by observations that can be independently reproduced and statistically validated. In other words, not all published findings are reliable.

In medicine, unreliable evidence can pose a risk to patients. Although the number of affected patients is difficult to quantify, the potential consequences can be substantial.

Drug Development

Drug development generally rests on two fundamental pillars: preclinical research and clinical trials in humans. Preclinical research precedes human clinical trials and provides evidence to inform their design and ethical justification.

Preclinical studies are conducted in the laboratory and include biochemical, genetic, cellular, and animal experiments. Reproducibility studies have shown that many preclinical findings cannot be replicated, raising questions regarding their validity. A striking example was reported in 2011, when researchers at Bayer attempted to validate published findings relevant to potential drug targets. Of the 67 target validation projects, nearly two thirds were discontinued because the company’s in-house findings did not match the published results.

Several systematic studies have reached similar conclusions. Possible explanations include inappropriate methods, instruments, or reagents, poorly representative cell lines, and animal models that do not adequately reflect human disease. Problems can also arise during data analysis, including false-positive findings resulting from type I errors, misinterpretation of results, biased interpretation, selective reporting, and arbitrary changes to the data. In the most serious cases, research misconduct may involve fabrication or falsification of data.

Clinical research also faces challenges. It requires substantial financial investment and, in most cases, is driven by the prospect of profit, creating strong incentives that can influence how the results are produced and communicated.

Studies conducted during the first decade of this century reinforced concerns that medical literature was not entirely reliable. Systematic reviews have shown that some clinical trials reported greater treatment effects than were subsequently observed or produced findings that were later contradicted by other studies. These concerns were more common in nonrandomized studies and those involving a small number of patients. In some cases, the trials were terminated before their scheduled end date as soon as a favorable and statistically significant result was observed.

Stopping a clinical trial early for benefit can protect participants from continuing to receive a placebo or inferior treatment once sufficient evidence shows that the intervention is effective. When a clinical trial is stopped early, there is a high risk that the effect size is exaggerated due to random statistical fluctuations.

Another form of misconduct is incomplete reporting of clinical trial results, including selective reporting of data, which can skew the findings toward statistical significance. Researchers may also change the objectives of a study after the fact. For example, a drug intended to treat heart failure may have no effect on the planned outcome, such as prolonging survival. Researchers could then focus on unexpected but statistically significant findings, such as a reduction in blood troponin concentrations, and present the trial as successful.

At least one study has shown that clinical trials sponsored by profit organizations are much more likely to recommend experimental drugs as the treatment of choice than trials funded by nonprofit organizations.

How to Combat These Practices

To address these questionable and potentially misleading practices, the International Committee of Medical Journal Editors (ICMJE) requires prospective clinical trial registration at or before the time of first patient enrollment as a condition of consideration for publication, a policy announced in 2005. In principle, preregistration makes it possible to verify that a study was conducted as planned and that the methods and outcomes of the study did not change during or after the study.

Preregistration of clinical trials has grown significantly since 2005 and is now much more widespread, although it is not yet universal. One survey found that, even in 2017, just over 60% of clinical trials had been registered.

Even after the 2005 mandate by the ICMJE, which required all clinical trials to be publicly registered before enrolling participants, numerous trials conducted after 2005 have continued to show selective reporting and changes that favor positive results, with substantial discrepancies between preregistered protocols and subsequently published findings.

For preregistration to be effective, study authors, peer reviewers, and journal editors must carry out the necessary checks, but this does not always happen. Nevertheless, there are indications that the accuracy of published results has improved since the introduction of preregistration and other transparency measures.

Should we be concerned about practices that undermine science, mislead physicians, and have serious consequences for patients? In my opinion, yes. At the same time, clinicians need a practical way to navigate the complex landscape of medical information without becoming overwhelmed by its volume, complexity, and inconsistencies.

Without claiming to be exhaustive or offering any guarantees, there are a few practical rules that may help:

• Whenever possible, give greater weight to randomized studies with an adequate number of patients.
• Consider the sponsors of scientific studies without treating sponsorship as a decisive factor.
• View isolated findings with healthy skepticism, particularly when they are unexpected findings, and, whenever possible, wait for confirmation from independent studies.
• When available, rely on high-quality meta-analyses, which can help identify the most reliable evidence and draw conclusions from the strongest findings.
• Newly introduced drugs may exhibit drug interactions and adverse effects that have not yet been identified. When efficacy is comparable, it may be preferable to use alternatives with an established safety record.
• Do not rely too heavily on information provided by pharmaceutical companies because financial interests may influence the information they provide.

Marco Crescenzi, MD, PhD, is a physician and former research director at the Italian National Institute of Health in Rome, Italy. During his career, he conducted extensive research at Washington University in St. Louis; the National Institutes of Health in Bethesda, Maryland; Sapienza University of Rome, Rome, Italy; and Regina Elena National Cancer Institute in Rome, Italy. He has conducted pioneering studies on cell proliferation and differentiation in regenerative medicine and is a passionate educator and scientific communicator.

This story was translated from Univadis Italy, part of the Medscape Professional Network.