Type I Error Rates and Power of Multiple Hypothesis Testing Procedures in Factorial ANOVA

Abstract

There are numerous General Linear Model (GLM) statistical designs that may require multiple hypothesis testing (MHT) procedures that control the Type I error inflation that occurs with multiple tests. This study investigated familywise error rates (FWER) and statistical power rates of several alpha-adjustment MHT procedures in factorial ANOVA, but results apply broadly to GLM procedures. Of four MHT procedures investigated, the Hochberg procedure performed most efficiently in terms of Type I error and power, slightly better than Holm. The Holm procedure, however, may be the better choice because of less restrictive assumptions. FWER concerns were raised with the Benjamini-Hochberg procedure.