In this essay I discuss potential outcome and graphical approaches to causality, and their relevance for empirical work in economics. I review some of the work on directed acyclic graphs, including the recent The Book of Why (Pearl and Mackenzie 2018). I also discuss the potential outcome framework developed by Rubin and coauthors (e.g., Rubin 2006), building on work by Neyman (1990 [1923]). I then discuss the relative merits of these approaches for empirical work in economics, focusing on the questions each framework answers well, and why much of the the work in economics is closer in spirit to the potential outcome perspective.

Two recent papers, Deaton (2009) and Heckman and Urzua (2009), argue against what they see as an excessive and inappropriate use of experimental and quasi-experimental methods in empirical work in economics in the last decade. They specifically question the increased use of instrumental variables and natural experiments in labor economics and of randomized experiments in development economics. In these comments, I will make the case that this move toward shoring up the internal validity of estimates, and toward clarifying the description of the population these estimates are relevant for, has been important and beneficial in increasing the credibility of empirical work in economics. I also address some other concerns raised by the Deaton and Heckman–Urzua papers.

In this paper I discuss alternatives to the GMM estimators proposed by Hansen (1982) and others. These estimators are shown to have a number of advantages. First of all, there is no need to estimate in an initial step a weight matrix as required in the conventional estimation procedure. Second, it is straightforward to derive the distribution of the estimator under general misspecification. Third, some of the alternative estimators have appealing information-theoretic interpretations. In particular, one of the estimators is an empirical likelihood estimator with an interpretation as a discrete support maximum likelihood estimator. Fourth, in an empirical example one of the new estimators is shown to perform better than the conventional estimators. Finally, the new estimators make it easier for the researcher to get better approximations to their distributions using saddlepoint approximations. The main cost is computational: the system of equations that has to be solved is of greater dimension than the number of parameters of interest. In practice this may or may not be a problem in particular applications.

This essay describes the evolution and recent convergence of two methodological approaches to causal inference. The first one, in statistics, started with the analysis and design of randomized experiments. The second, in econometrics, focused on settings with economic agents making optimal choices. I argue that the local average treatment effects framework facilitated the recent convergence by making key assumptions transparent and intelligible to scholars in many fields. Looking ahead, I discuss recent developments in causal inference that combine the same transparency and relevance.

In this paper, a new estimator is proposed for discrete choice models with choice-based sampling. The estimator is efficient and can incorporate information on the marginal choice probabilities in a straightforward manner and for that case leads to a procedure that is computationally and intuitively more appealing than the estimators that have been proposed before. The idea is to start with a flexible parametrization of the distribution of the explanatory variables and then rewrite the estimator to remove dependence on these parametric assumptions.

Recently there has been a surge in econometric work focusing on estimating average treatment effects under various sets of assumptions. One strand of this literature has developed methods for estimating average treatment effects for a binary treatment under assumptions variously described as exogeneity, unconfoundedness, or selection on observables. The implication of these assumptions is that systematic (for example, average or distributional) differences in outcomes between treated and control units with the same values for the covariates are attributable to the treatment. Recent analysis has considered estimation and inference for average treatment effects under weaker assumptions than typical of the earlier literature by avoiding distributional and functional-form assumptions. Various methods of semiparametric estimation have been proposed, including estimating the unknown regression functions, matching, methods using the propensity score such as weighting and blocking, and combinations of these approaches. In this paper I review the state of this literature and discuss some of its unanswered questions, focusing in particular on the practical implementation of these methods, the plausibility of this exogeneity assumption in economic applications, the relative performance of the various semiparametric estimators when the key assumptions (unconfoundedness and overlap) are satisfied, alternative estimands such as quantile treatment effects, and alternate methods such as Bayesian inference.

In many empirical studies of the effect of social programs researchers assume that, conditional on a set of observed covariates, assignment to the treatment is exogenous or unconfounded (aka selection on observables). Often this assumption is not realistic, and researchers are concerned about the robustness of their results to departures from it. One approach (e.g., Charles Manski, 1990) is to entirely drop the exogeneity assumption and investigate what can be learned about treatment effects without it. With unbounded outcomes, and in the absence of alternative identifying assumptions, there are no restrictions on the set of possible values for average treatment effects. This does not mean, however, that all evaluations are equally sensitive to departures from the exogeneity assumption. In this paper I explore an alternative approach, developed by Paul Rosenbaum and Donald Rubin (1983), where the assumption of exogeneity is explicitly relaxed by allowing for a limited amount of correlation between treatment and unobserved components of the outcomes. The starting point of the sensitivity analysis is the assumption that the exogeneity assumption is satisfied only conditional on an additional unobserved covariate. Making assumptions about the effect of the unobserved covariate on the outcome and its correlation with the treatment, I trace out the set of possible values for the treatment effect of interest. By considering a sufficiently large set of possible correlations with outcomes and treatment, one can recover the bounds on the treatment effect derived by Manski (1990). The approach here, in the spirit of Rosenbaum and Rubin (1983) and Rosenbaum (1995), is to allow only a limited amount of correlation and to judge the sensitivity of average treatment-effect estimates to such correlations. There are two novel features of the proposed analysis. First, rather than formulate the sensitivity in terms of coefficients on the unobserved covariate, the sensitivity results are presented in terms of partial R values, which may be easier to interpret. Second, the partial R values of the unobserved covariates are compared to those for the observed covariates in order to facilitate judgments regarding the plausibility of values necessary to substantially change results obtained under exogeneity. The proposed sensitivity analysis is conceptually related to the practice of assessing sensitivity of estimates by comparisons with results obtained by discarding one or more observed covariates (James Heckman and V. Joseph Hotz, 1989; Rajeev Dehejia and Sadek Wahba, 1999; Jeffrey Smith and Petra Todd, 2001). The attraction of the sensitivity analysis is that it is more directly relevant: one is not interested in what would have happened in the absence of covariates actually observed, but in biases that are the result from not observing all relevant covariates.

We develop a new approach for estimating average treatment effects in observational studies with unobserved group-level heterogeneity. We consider a general model with group-level unconfoundedness and provide conditions under which aggregate balancing statistics—group-level averages of functions of treatments and covariates—are sufficient to eliminate differences between groups. Building on these results, we re-interpret commonly used linear fixed-effect regression estimators by writing them in the Mundlak form as linear regression estimators without fixed effects but including group averages. We use this representation to develop Generalized Mundlak Estimators that capture group differences through group averages of (functions of) the unit-level variables and adjust for these group differences in flexible and robust ways in the spirit of the modern causal literature.

Propensity score matching estimators (Rosenbaum and Rubin (1983)) are widely used in evaluation research to estimate average treatment effects. In this article, we derive the large sample distribution of propensity score matching estimators. Our derivations take into account that the propensity score is itself estimated in a first step, prior to matching. We prove that first step estimation of the propensity score affects the large sample distribution of propensity score matching estimators, and derive adjustments to the large sample variances of propensity score matching estimators of the average treatment effect (ATE) and the average treatment effect on the treated (ATET). The adjustment for the ATE estimator is negative (or zero in some special cases), implying that matching on the estimated propensity score is more efficient than matching on the true propensity score in large samples. However, for the ATET estimator, the sign of the adjustment term depends on the data generating process, and ignoring the estimation error in the propensity score may lead to confidence intervals that are either too large or too small.

Matching estimators are widely used in empirical economics for the evaluation of programs or treatments. Researchers using matching methods often apply the bootstrap to calculate the standard errors. However, no formal justification has been provided for the use of the bootstrap in this setting. In this article, we show that the standard bootstrap is, in general, not valid for matching estimators, even in the simple case with a single continuous covariate where the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias. Valid inferential methods in this setting are the analytic asymptotic variance estimator of Abadie and Imbens (2006a) as well as certain modifications of the standard bootstrap, like the subsampling methods in Politis and Romano (1994).