Affordable Access

Access to the full text

A P-value model for theoretical power analysis and its applications in multiple testing procedures

  • Zhang, Fengqing1
  • Gou, Jiangtao2
  • 1 Drexel University, Department of Psychology, 3201 Chestnut Street, Philadelphia, 19104, USA , Philadelphia (United States)
  • 2 Hunter College of CUNY, Department of Mathematics and Statistics, 695 Park Avenue, New York, 10065, USA , New York (United States)
Published Article
BMC Medical Research Methodology
Springer (Biomed Central Ltd.)
Publication Date
Oct 10, 2016
DOI: 10.1186/s12874-016-0233-0
Springer Nature


BackgroundPower analysis is a critical aspect of the design of experiments to detect an effect of a given size. When multiple hypotheses are tested simultaneously, multiplicity adjustments to p-values should be taken into account in power analysis. There are a limited number of studies on power analysis in multiple testing procedures. For some methods, the theoretical analysis is difficult and extensive numerical simulations are often needed, while other methods oversimplify the information under the alternative hypothesis. To this end, this paper aims to develop a new statistical model for power analysis in multiple testing procedures.MethodsWe propose a step-function-based p-value model under the alternative hypothesis, which is simple enough to perform power analysis without simulations, but not too simple to lose the information from the alternative hypothesis. The first step is to transform distributions of different test statistics (e.g., t, chi-square or F) to distributions of corresponding p-values. We then use a step function to approximate each of the p-value’s distributions by matching the mean and variance. Lastly, the step-function-based p-value model can be used for theoretical power analysis.ResultsThe proposed model is applied to problems in multiple testing procedures. We first show how the most powerful critical constants can be chosen using the step-function-based p-value model. Our model is then applied to the field of multiple testing procedures to explain the assumption of monotonicity of the critical constants. Lastly, we apply our model to a behavioral weight loss and maintenance study to select the optimal critical constants.ConclusionsThe proposed model is easy to implement and preserves the information from the alternative hypothesis.

Report this publication


Seen <100 times