Journal of Finance Vol. 78 No. 4 2023

Reusing Natural Experiments

Davidson Heath^{1,2,3,4,5,6,7,8,9}; Matthew C. Ringgenberg^{1,2,3,4,5,6,7,8,9}; Mehrdad Samadi^{1,2,3,4,5,6,7,8,9}; Ingrid M. Werner^{1,2,3,4,5,6,7,8,9}

¹ Rutgers, The State University of New Jersey · ² United States Securities and Exchange Commission · ³ Federal Reserve · ⁴ University of Utah · ⁵ Federal Reserve Board of Governors · ⁶ University of Illinois Chicago · ⁷ University of Virginia · ⁸ The Ohio State University · ⁹ Texas A&M University

Abstract

After a natural experiment is first used, other researchers often reuse the setting, examining different outcome variables. We use simulations based on real data to illustrate the multiple hypothesis testing problem that arises when researchers reuse natural experiments. We then provide guidance for future inference based on popular empirical settings including difference‐in‐differences, instrumental variables, and regression discontinuity designs. When we apply our guidance to two extensively studied natural experiments, business combination laws and the Regulation SHO pilot, we find that many results that were statistically significant using single hypothesis testing do not survive corrections for multiple hypothesis testing.

DOI

10.1111/jofi.13250

Volume

78

Issue

4

Pages

2329-2364

Language

en

Sources

bibtex:phds-export.bib openalex crossref

Cite

APA 7 MLA 9 Chicago (author–date) Harvard Elsevier Harvard (JFE, JBF…) The Accounting Review Academy of Management American Marketing Association INFORMS (Management Science) Copy citation Copied ✓