Affordable Access

Access to the full text

Why statistical inference from clinical trials is likely to generate false and irreproducible results

  • Hanin, Leonid1, 2
  • 1 Idaho State University, Department of Mathematics and Statistics, 921 S. 8th Avenue, Stop 8085, Pocatello, ID, 83209-8085, USA , Pocatello (United States)
  • 2 St. Petersburg Polytechnic University, Department of Applied Mathematics, Institute of Mathematics and Mechanics, Polytekhnicheskaya ul. 29, St. Petersburg, 195251, Russia , St. Petersburg (Russia)
Published Article
BMC Medical Research Methodology
Springer (Biomed Central Ltd.)
Publication Date
Aug 22, 2017
DOI: 10.1186/s12874-017-0399-0
Springer Nature


One area of biomedical research where the replication crisis is most visible and consequential is clinical trials. Why do outcomes of so many clinical trials contradict each other? Why is the effectiveness of many drugs and other medical interventions so low? Why have prescription medications become the third leading cause of death in the US and Europe after cardiovascular diseases and cancer? In answering these questions, the main culprits identified so far have been various biases and conflicts of interest in planning, execution and analysis of clinical trials as well as reporting their outcomes. In this work, we take an in-depth look at statistical methodology used in planning clinical trials and analyzing trial data. We argue that this methodology is based on various questionable and empirically untestable assumptions, dubious approximations and arbitrary thresholds, and that it is deficient in many other respects. The most objectionable among these assumptions is that of distributional homogeneity of subjects’ responses to medical interventions. We analyze this and other assumptions both theoretically and through clinical examples. Our main conclusion is that even a totally unbiased, perfectly randomized, reliably blinded, and faithfully executed clinical trial may still generate false and irreproducible results. We also formulate a few recommendations for the improvement of the design and statistical methodology of clinical trials informed by our analysis.

Report this publication


Seen <100 times