Challenges to Replication and Iteration in Field Experiments: Evidence from Two Direct Mail Shots

Jake Bowers; Nathaniel Higgins; Dean Karlan; Sarah Tulman; Jonathan Zinman

doi:10.1257/aer.p20171060

Challenges to Replication and Iteration in Field Experiments: Evidence from Two Direct Mail Shots

Jake Bowers¹; Nathaniel Higgins²; Dean Karlan³; Sarah Tulman⁴; Jonathan Zinman⁵

¹ University of Illinois, 420 David Kinley Hall, 1407 W Gregory Drive, Urbana, IL 61801, and SBST (e-mail: ) · ² Social and Behavioral Sciences Team, 1650 Pennsylvania Avenue NW, Washington, DC 20502 (e-mail: ) · ³ Yale University, 27 Hillhouse Avenue, New Haven, CT 06511 (e-mail: ) · ⁴ US Department of Agriculture, Economic Research Service, 1400 Independence Avenue SW, Mail Stop 1800, Washington, DC 200250-0002 (e-mail: ) · ⁵ Dartmouth College, HB6106, Hanover, NH 03755 (e-mail: )

American Economic Review 2017

We conducted an experiment marketing microloans to farmers in the USA during Spring 2015 and found a simple direct mail letter increased borrowing from a government program. The subsequent spring, we built on this finding and enriched the design to test for information spillovers. The direct effect result did not replicate in the second year, thus lowering the likelihood that spillovers would be present and detectable. These results add to recent evidence on how (seemingly subtle) differences in context and treatment content affect consumer responses.

DOI: 10.1257/aer.p20171060
Volume: 107 (5)
Pages: 462-465
Language: en
Export: BibTeX
Sources: openalex crossref