Performance of the Roy-Bargmann Stepdown Procedure as a Follow Up to a Significant MANOVA

Abstract

The Roy-Bargmann procedure has been suggested as a post hoc procedure for a significant MANOVA result. This method, which is based on application of the univariate General Linear Model, requires the researcher to order the dependent variables a priori in terms of their contextual importance. Subsequently, if the MANOVA is found to be statistically significant for a categorical independent
variable, these response variables are tested individually using univariate analyses in the sequence established a priori. Thus, the first variable is treated as the dependent variable in an ANOVA and if the groups’ means are found to differ significantly on this variable, it serves as a covariate while the second variable in the sequence is the response and the means of the groups are once again compared. This testing continues for each of the response variables, with variables higher in the sequence serving as covariates for those lower in importance. The current study was designed to examine the Type I error rate and power of this post hoc approach under a variety of data conditions. Results show that factors such as distribution of the dependent variables, equality (or lack thereof) of the covariance matrices and sample size all have a significant impact on Type I error and power. Furthermore, both Type I error rates and power of variables later in the sequence are influenced by variables earlier in the sequence.