← Search

Review of Financial Studies Vol. 37 No. 11 2024

Computational Reproducibility in Finance: Evidence from 1,000 Tests

Christophe Pérignon1; Olivier Akmansoy2; Christophe Hurlin3; Anna Dreber4; Felix Holzmeister5; Jürgen Huber5; Magnus Johannesson6; Michael Kirchler5; Albert J. Menkveld7; Michael Razen5; Utz Weitzel8

1 HEC Paris , , and cascad, France · 2 CNRS , , and cascad, France · 3 University of Orléans , , and cascad, France · 4 Stockholm School of Economics , , and University of Innsbruck, Austria · 5 University of Innsbruck · 6 Stockholm School of Economics · 7 Vrije Universiteit Amsterdam, Netherlands, and Tinbergen Institute , · 8 Vrije Universiteit Amsterdam , , Radboud University, Netherlands, and Tinbergen Institute, Netherlands

Abstract

We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers’ code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and those exerting more effort. It is lower for more technical research questions, more complex code, and results lying in the tails of the distribution. Researchers exhibit overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss implementable reproducibility policies for academic journals.

DOI
10.1093/rfs/hhae029
Volume
37
Issue
11
Pages
3558-3593
Language
en
Sources
bibtex:phds-export.bib openalex crossref

Cite