A Comment on:“On the Informativeness of Descriptive Statistics for Structural Estimates” by Isaiah Andrews, Matthew Gentzkow, and Jesse M. Shapiro
ANDREWS, GENTZKOW, AND SHAPIRO (2020), henceforth AGS, make important progress on a challenging and fundamental question. The authors consider a setting in which a researcher reports an estimate ĉn that is unbiased under the researcher’s model. The researcher also reports a statistic γ̂n and a measure of informativeness of γ̂n for ĉn, denoted by . AGS show can be related to properties of the distribution of ĉn under models larger than the one employed by the researcher. As a result, can play a role in reassuring a reader who is concerned about some aspects of the researcher’s model. I illustrate the properties of with two examples. The first example is based on the classic selection problem of Heckman (1979) and portrays a setting in which a high is not reassuring. The example shows that if a researcher has assumed random assignment and the reader is concerned about it, then the bias of ĉn under the reader’s model can be arbitrarily large regardless of . The second example is based on the discussion of Attanasio, Meghir, and Santiago (2012) in AGS. This example illustrates that if ĉn and γ̂n are estimators of a common parameter, then can be linked to the properties of a Hausman (1978) test. As a result, √ 1 − can be shown to equal the (normalized) largest bias of ĉn under the reader’s model and, in this sense, a high is reassuring. I show this conclusion holds generally in a classical local asymptotic framework (Bickel, Klassen, Ritov, and Wellner (1993)). The two examples underscore that, as emphasized by AGS, whether a high is reassuring to the reader can crucially depend on the particulars of the researcher’s and reader’s models. I employ the notation in AGS with some minor modifications. The researcher’s model is indexed by η ∈ H. The reader entertains a larger model indexed by (η ζ) ∈ H × Z. I focus on the perspective of the reader, who views the parameter of interest as a function of (η ζ) that we denote by c(η ζ). Under the reader’s model, the distribution of the data is determined by the value of (η ζ). To emphasize this dependence, I write E(η ζ)[ĉn] to denote the expectation of ĉn under (η ζ). The property that ĉn is unbiased under the researcher’s model is equivalent to E(η 0)[ĉn] = c(η 0) for any η ∈H, while the bias under (η ζ) in the reader’s model is given by E(η ζ)[ĉn] − c(η ζ). As in AGS, I consider an analysis local to proper specification, meaning we study (η ζ) that are close to some (η0 0). Since E(η0 0)[ĉn] = c(η0 0) due to ĉn being unbiased under the researcher’s model, the bias of ĉn under any (η ζ) in the reader’s model equals E(η ζ)[ĉn] − c(η ζ)