Human-Friendly Scheffé Comparisons, or the Art of Complex Multiple Comparisons

Abstract

Researchers learn that the Scheffé (1953) MCP lacks power because it adjusts for all possible comparisons—consequently, few use it. However, only Scheffé guarantees congruence: finding a significant comparison when the omnibus ANOVA is significant—and not finding one when ANOVA is nonsignificant (Maxwell et al., 2018). A maximum Scheffé comparison can be calculated to provide the set of coefficients that maximally differentiates some combination of groups on the dependent variable (Keppel & Wickens, 2004; Williams, 1978). Unfortunately, coefficient weights from this maximum comparison are often uninterpretable. Therefore, we have developed a Shiny app to identify maximum and other Barcikowski “human-friendly” comparisons that may actually be meaningful. We used Monte Carlo simulations to investigate robustness, power, and congruence (Kirk, 2013) of these human-friendly comparisons and the relatively unknown Brown-Forsythe unequal-variance adjustment to Scheffé. We report results regarding Bonferroni-adjusted normality tests and zero-adjusted Levene homoscedasticity tests for assumptions in ANOVA.