The p-Problem with Forward Selection Stepwise Regression: Algorithm for Controlling Type I Errors

Abstract

The use of forward selection stepwise regression has been criticized for both interpretive misunderstandings and statistical aberrations. A major statistical problem with stepwise regression and other procedures that involve multiple significance tests is the inflation of the Type I error rate. Common approaches to control the family-wise error rate (e.g., the Bonferroni and Sidak corrections) are based on the assumptions of independent tests which typically reduce power. Because the presence of correlated predictors is a more realistic situation, other algorithms based on the average correlation in the predictor matrix have been proposed. The present study proposes an algorithm based on the maximum eigenvalue and the determinant of the predictor matrix for controlling the familywise Type I error rate for multiple, correlated tests in forward selection regression under the complete null hypothesis. A Monte Carlo simulation with 5,000 replications was performed to demonstrate the effectiveness of the proposed algorithm.