The Use of Propensity Score Analysis to Address Issues Associated with the Use of Adjusted Means Produced by Analysis of Covariance

Abstract

It is common for researchers in the field of education to engage in research that involves two groups (e.g., control and experimental) and not have the opportunity to randomly assign the participants to the groups. The challenge facing educational researchers is how to analyze the differences between two groups when randomization is not possible and selection bias is an issue. An analytic technique commonly used by educational researchers to address this challenge is analysis of covariance. The use of this technique,
however, raises two concerns: (a) The inclusion of the covariates in the analysis of the criterion variable may change the construct represented by the criterion variable, and (b) the analytic technique employed does not match the research question, which is a Type VI error. A technique referred to as propensity score analysis, which is also designed to deal with selection bias in the comparison of non-randomized group means, may address these two concerns. How this technique can be applied by educational researchers is presented.