Performance of OLS and HCCM Estimators in Heteroscedastic ANCOVA Models

Abstract

We simulated several heteroscedastic scenarios for an Analysis of Covariance (ANCOVA) model with an orthogonal fixed covariate. We examined the Type 1 and 2 error rates of OLS and Heteroscedastic Consistent Covariance Matrix (HCCM) estimators for detecting adjusted mean differences and covariate effects when heteroscedasticity was a function of differences in group variances, the covariate, and both processes in unbalanced ANCOVA models. Results indicate that under complete null models, heteroscedasticity due to an orthogonal covariate alone did not affect the Type 1 error rate for the OLS test of adjusted group mean differences; however, heteroscedasticity due to both the covariate and group, can attenuate, exacerbate, or reverse the known effects of heteroscedasticity on test size in unbalanced models. Furthermore, heteroscedasticity due to the covariate drastically affected the power of the tests of the group effect. HC2 and HC3 tests of the group effect were the most powerful among tests with valid test size; however, both HC2 and HC3 tests for the covariate effect inflated the Type 1 error rate. The C²_(H) test proposed by Cai and Hayes (2008) was promising with Type 1 error rates held below the nominal alpha for both tests of the group and covariate effects in all conditions simulated; however, it was conservative in terms of statistical power.