Regression-Discontinuity with Nonparametric Bootstrap

Abstract

The regression-discontinuity design (RD) is a powerful methodological alternative to the quasi-experimental design when conducting evaluations. RD designs involve testing post-test mean treatment differences between the experimental and comparison group regression lines at the centered cutoff point for statistical significance. This study simulated a RD treatment effect of 7 points in simulated normal and non-normal data distributions. The bootstrap technique was then used to estimate stability of estimates. Evaluation data oftentimes is non-normal, so understanding whether this impacts the RD design analysis is important. The bias (difference between the observed treatment effect and bootstrap estimate) and the confidence intervals are reported. Bootstrap estimates are useful in understanding whether the treatment effect is stable and the amount of estimation error present in RD given underlying normal and non-normal distributions. Results indicated that estimates of RD treatment effects are not severely impacted by non-normal, positive skewed, distributions. Consequently, robust estimation methods and/or data transformations such as probit are most likely sufficient to provide accurate stable estimates of treatment effects when concerned about meeting assumptions in regression analyses.