Multivariate Regression with Small Samples: A Comparison of Estimation Methods

Abstract

High dimensional multivariate data, where the number of variables approaches or exceeds the sample size, is an increasingly common occurrence for social scientists. Several tools exist for dealing with such data in the context of univariate regression, including regularization methods (i.e., Lasso, Elastic net, Ridge Regression, as well as Bayesian models with spike and slab priors. These methods have not been widely studied in the context of multivariate regression modeling. Thus, the goal of this simulation study was to compare the performance of these methods for high dimensional data with multivariate regression, in which there exist more than one dependent variable. Simulation results revealed that the regularization methods, particularly Ridge Regression, were found to be particularly effective in terms of parameter estimation accuracy and control over the Type I error rate. Implications for practice are discussed.