We study the effectiveness of nonexperimental strategies in adjusting for comparison group differences when using data from several programs, each implemented at a different location, to compare their effect if implemented at alternative locations. First, we adjust for individual characteristics differences simultaneously across all groups using unconfoundedness-based and conditional difference-in-difference methods for multiple treatments. Second, we adjust for differences in local economic conditions and stress their role after program participation. Our results show that it is critical to have sufficient overlap across locations in both dimensions and illustrate the difficulty of adjusting for local economic conditions that differ greatly across locations.

In this paper we develop two nonparametric tests of treatment effect heterogeneity. The first test is for the null hypothesis that the treatment has a zero average effect for all subpopulations defined by covariates. The second test is for the null hypothesis that the average effect conditional on the covariates is identical for all subpopulations, that is, that there is no heterogeneity in average treatment effects by covariates. We derive tests that are straightforward to implement and illustrate the use of these tests on data from two sets of experimental evaluations of the effects of welfare-to-work programs.