Control of false positives in randomized phase III clinical trials

Randomized Phase III clinical trials serve as the gold-standard for the evaluation of the efficacy of a medical intervention. Although research and development in earlier stages together with rigorous statistical examination assure a small probability of false positive for a given trial, it is unclear how many false positives were generated from the large number of randomized Phase III trials from the biopharmaceutical industry in the United States. The proportion of comparisons in Phase III trials where the medical intervention has null or negative efficacy, or proportion of null or negative (PNN), is at the central position for the estimation and control of the number of false positives. We seek to estimate PNN using a new Bayesian deconvolution method. Using data from clinicaltrials.gov and other data sources, we identified 1393 trials completed in 2008-2012 that meet our study entry criteria, which are dominated by trials on drugs for treatment purpose. Among the 1221 trials with results available on the selected comparisons, 789 (64.6%) show statistically significant superiority of the intervention, with 561 (45.9%) having a two-sided p-value less than 0.001. The PNN is estimated to be no more than 7-9%. Based on the PNN, we estimated that 18-22% of the trials have at least one comparison with null or negative efficacy, leading to an expectation of no more than 6-8 trials with at least one false positive comparison over a 5-year period.