A Fast Literature Search Engine based on top-quality journals, by Dr. Mingze Gao.
- Topic classification is ongoing.
- Please kindly let me know [mingze.gao@mq.edu.au] in case of any errors.
Machine Learning as a Tool for Hypothesis Generation*
Resource type
Authors/contributors
- Ludwig, Jens (Author)
- Mullainathan, Sendhil (Author)
Title
Machine Learning as a Tool for Hypothesis Generation*
Abstract
While hypothesis testing is a highly formalized activity, hypothesis generation remains largely informal. We propose a systematic procedure to generate novel hypotheses about human behavior, which uses the capacity of machine learning algorithms to notice patterns people might not. We illustrate the procedure with a concrete application: judge decisions about whom to jail. We begin with a striking fact: the defendant’s face alone matters greatly for the judge’s jailing decision. In fact, an algorithm given only the pixels in the defendant’s mug shot accounts for up to half of the predictable variation. We develop a procedure that allows human subjects to interact with this black-box algorithm to produce hypotheses about what in the face influences judge decisions. The procedure generates hypotheses that are both interpretable and novel: they are not explained by demographics (e.g., race) or existing psychology research, nor are they already known (even if tacitly) to people or experts. Though these results are specific, our procedure is general. It provides a way to produce novel, interpretable hypotheses from any high-dimensional data set (e.g., cell phones, satellites, online behavior, news headlines, corporate filings, and high-frequency time series). A central tenet of our article is that hypothesis generation is a valuable activity, and we hope this encourages future work in this largely “prescientific” stage of science.
Publication
The Quarterly Journal of Economics
Volume
139
Issue
2
Pages
751-827
Date
2024-05-01
Journal Abbr
The Quarterly Journal of Economics
ISSN
0033-5533
Accessed
6/28/24, 1:01 PM
Library Catalog
Silverchair
Citation
Ludwig, J., & Mullainathan, S. (2024). Machine Learning as a Tool for Hypothesis Generation*. The Quarterly Journal of Economics, 139, 751–827.
Link to this record