Dimension Reduction Regression Techniques for High Dimensional Data

Abstract

In some situations, researchers are faced with a regression analysis involving a large number of independent variables (predictors) relative to the sample size. Such cases are referred to as high dimensional data and can present problems for standard regression analyses, including collinearity among the independent variables, and in the extreme case an inability to obtain parameter estimates. One data analysis strategy that has been recommended for such situations is principal components analysis, which combines the predictors into a smaller number of linear combinations that are then used as independent variables themselves in a regression analysis with the original dependent measure. Recently, several variations to this approach for data reduction have been recommended for use. The goal of this paper was to describe several of these and to innovate their use with high dimensional social science datasets.