Unbalanced Sampling Effect on the Power at Level-1 in the Random Coefficient Model

Abstract

Researchers often disregard the potentially negative effects of unbalanced sampling on power estimates when using multilevel models. The purpose of this study was to investigate the effects that unbalanced sampling had on the estimated level-one power in multilevel random coefficient models. Twelve combinations of three effect sizes (0.5, 0.8, and 1.0) and four intraclass correlations (0.2, 0.1, 0.05, and 0.01) were investigated with each of three sampling ratios (0.25:0.75, 0.20:0.80, and 0.15:0.85) and three sample sizes (200, 500, and 800) to compare the effects that the different sampling ratios had on the level-1 power in the random coefficient model. Results indicated that as sampling ratios changed from 0.25:0.75, to incrementally a larger unbalanced sampling ratio of 0.15:0.85, the estimated power was lower in almost every case. This effect was more pronounced for the smaller sample sizes. Fourteen cases displayed differences larger than 5% in aggregate power estimates.