We develop a new approach to estimating the causal effects of treatments or instruments that combine multiple sources of variation according to a known formula. Examples include treatments capturing spillovers in social or transportation networks and simulated instruments for policy eligibility. We show how exogenous shocks to some, but not all, determinants of such variables can be leveraged while avoiding omitted variables bias. Our solution involves specifying counterfactual shocks that may as well have been realized and adjusting for a summary measure of non-randomness in shock exposure: the average treatment (or instrument) across shock counterfactuals. We use this approach to address bias when estimating employment effects of market access growth from Chinese high-speed rail construction.

Many studies use shift-share (or "Bartik") instruments, which average a set of shocks with exposure share weights. We provide a new econometric framework for shift-share instrumental variable (SSIV) regressions in which identification follows from the quasi-random assignment of shocks, while exposure shares are allowed to be endogenous. The framework is motivated by an equivalence result: the orthogonality between a shift-share instrument and an unobserved residual can be represented as the orthogonality between the underlying shocks and a shock-level unobservable. SSIV regression coefficients can similarly be obtained from an equivalent shock-level regression, motivating shock-level conditions for their consistency. We discuss and illustrate several practical insights of this framework in the setting of Autor et al. (2013), estimating the effect of Chinese import competition on manufacturing employment across U.S. commuting zones.

We develop new quasi-experimental tools to measure disparate impact, regardless of its source, in the context of bail decisions. We show that omitted variables bias in pretrial release rate comparisons can be purged by using the quasi-random assignment of judges to estimate average pretrial misconduct risk by race. We find that two-thirds of the release rate disparity between white and Black defendants in New York City is due to the disparate impact of release decisions. We then develop a hierarchical marginal treatment effect model to study the drivers of disparate impact, finding evidence of both racial bias and statistical discrimination.

Abstract A long-standing and influential view in U.S. correctional policy is that “nothing works” when it comes to rehabilitating incarcerated individuals. We revisit this hypothesis by studying an innovative law-enforcement-led program launched in the county jail of Flint, MI: Inmate Growth Naturally and Intentionally through Education (IGNITE). We develop an instrumental variables approach to estimate the effects of IGNITE exposure, leveraging quasi-random court delays that cause individuals to spend more time in jail before and after the program’s launch. Holding time in jail fixed, we find that one additional month of IGNITE exposure reduces weekly misconduct in jail by 25% and three-month recidivism by 24%, with the recidivism effects growing over time. Surveys of staff and community members, along with administrative test-score records and within-jail text messages, suggest that cultural change and improved literacy and numeracy scores are contributing mechanisms.

Competition in health insurance markets may fail to improve health outcomes if consumers are not able to identify high quality plans. We develop and apply a novel instrumental variables framework to quantify the variation in causal mortality effects across plans and how much consumers attend to this variation. We first document large differences in the observed mortality rates of Medicare Advantage plans within local markets. We then show that when plans with high (low) mortality rates exit these markets, enrollees tend to switch to more typical plans and subsequently experience lower (higher) mortality. We derive and validate a novel "fallback condition" governing the subsequent choices of those affected by plan exits. When the fallback condition is satisfied, plan terminations can be used to estimate the relationship between observed plan mortality rates and causal mortality effects. Applying the framework, we find that mortality rates unbiasedly predict causal mortality effects. We then extend our framework to study other predictors of plan mortality effects and estimate consumer willingness to pay. Higher spending plans tend to reduce enrollee mortality, but existing quality ratings are uncorrelated with plan mortality effects. Consumers place little weight on mortality effects when choosing plans. Good insurance plans dramatically reduce mortality, and redirecting consumers to such plans could improve beneficiary health.

Abstract We introduce two empirical strategies harnessing the randomness in school assignment mechanisms to measure school value-added. The first estimator controls for the probability of school assignment, treating take-up as ignorable. We test this assumption using randomness in assignments. The second approach uses assignments as instrumental variables (IVs) for low-dimensional models of value-added and forms empirical Bayes posteriors from these IV estimates. Both strategies solve the underidentification challenge arising from school undersubscription. Models controlling for assignment risk and lagged achievement in Denver and New York City yield reliable value-added estimates. Estimates from models with lower-quality achievement controls are improved by IV.

We develop over-identification tests that use admissions lotteries to assess the predictive value of regression-based value-added models (VAMs). These tests have degrees of freedom equal to the number of quasi-experiments available to estimate school effects. By contrast, previously implemented VAM validation strategies look at a single restriction only, sometimes said to measure forecast bias. Tests of forecast bias may be misleading when the test statistic is constructed from many lotteries or quasi-experiments, some of which have weak first stage effects on school attendance. The theory developed here is applied to data from the Charlotte-Mecklenberg School district analyzed by Deming (2014).

Abstract Conventional value-added models (VAMs) compare average test scores across schools after regression-adjusting for students’ demographic characteristics and previous scores. This article tests for VAM bias using a procedure that asks whether VAM estimates accurately predict the achievement consequences of random assignment to specific schools. Test results from admissions lotteries in Boston suggest conventional VAM estimates are biased, a finding that motivates the development of a hierarchical model describing the joint distribution of school value-added, bias, and lottery compliance. We use this model to assess the substantive importance of bias in conventional VAM estimates and to construct hybrid value-added estimates that optimally combine ordinary least squares and lottery-based estimates of VAM parameters. The hybrid estimation strategy provides a general recipe for combining nonexperimental and quasi-experimental estimates. While still biased, hybrid school value-added estimates have lower mean squared error than conventional VAM estimates. Simulations calibrated to the Boston data show that, bias notwithstanding, policy decisions based on conventional VAMs that control for lagged achievement are likely to generate substantial achievement gains. Hybrid estimates that incorporate lotteries yield further gains.

We study regressions with multiple treatments and a set of controls that is flexible enough to purge omitted variable bias. We show these regressions generally fail to estimate convex averages of heterogeneous treatment effects—instead, estimates of each treatment’s effect are contaminated by nonconvex averages of the effects of other treatments. We discuss three estimation approaches that avoid such contamination bias, including the targeting of easiest-to-estimate weighted average effects. A reanalysis of nine empirical applications finds economically and statistically meaningful contamination bias in observational studies; contamination bias in experimental studies is more limited due to smaller variability in propensity scores. (JEL C21, C31, C51, H75, I21, I28)

Abstract Economists often measure discrimination as disparities arising from the direct effects of group identity. We develop new tools to model and measure systemic discrimination, capturing how discrimination in other decisions indirectly contributes to disparities. A novel experimental design, the iterated audit, identifies systemic discrimination. We illustrate these new tools in two field experiments. The first experiment shows how racial discrimination can accumulate across multiple rounds of hiring through the interaction of two forces: greater discrimination against inexperienced workers, which affects the opportunity to obtain experience, and high subsequent returns to experience. The second experiment shows how gender-based differences in the language of recommendation letters can translate into systemic gender discrimination in STEM hiring. We discuss how our findings qualify previous results on direct discrimination and how our tools can be used to target policy interventions.