“Site selection bias” can occur when the probability that a program is adopted or evaluated is correlated with its impacts. I test for site selection bias in the context of the Opower energy conservation programs, using 111 randomized control trials involving 8.6 million households across the United States. Predictions based on rich microdata from the first 10 replications substantially overstate efficacy in the next 101 sites. Several mechanisms caused this positive selection. For example, utilities in more environmentalist areas are more likely to adopt the program, and their customers are more responsive to the treatment. Also, because utilities initially target treatment at higher-usage consumer subpopulations, efficacy drops as the program is later expanded. The results illustrate how program evaluations can still give systematically biased out-of-sample predictions, even after many replications.

Imperfect information and inattention to energy costs are important potential motivations for energy efficiency standards and subsidies. We evaluate these motivations in the lightbulb market using a theoretical model and two randomized experiments. We derive welfare effects as functions of reduced-form sufficient statistics capturing economic and psychological parameters, which we estimate using a novel within-subject information disclosure experiment. The main results suggest that moderate subsidies for energy-efficient lightbulbs may increase welfare, but informational and attentional biases alone do not justify a ban on incandescent lightbulbs. Our results and techniques generate broader methodological insights into welfare analysis with misoptimizing consumers.

A corrective tax or subsidy is “well-targeted” if it primarily affects choices that are more distorted by market failures. Energy efficiency subsidies are designed to correct multiple distortions: externalities, credit constraints, “landlord-tenant” information asymmetries, imperfect information, and inattention. We show that three important energy efficiency subsidies are primarily taken up by consumers who are wealthier, own their own homes, and are more informed about and attentive to energy costs. This suggests that these subsidies are poorly targeted at the market failures they were designed to address. However, we show that “tagging” can lead to large efficiency gains.