Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.

We examine the nature of consumer demand for government-controlled online news outlets in Russia, testing whether such demand reflects a preference for progovernment ideological coverage or other factors unrelated to outlets’ ideological positions. We detect government-sensitive topics and measure outlets’ news-reporting decisions from news article texts, and we estimate a structural model of demand for news, using detailed browsing data that traces individual-level consumption. The average consumer has a distaste for progovernment ideology but a strong, persistent taste for state-owned outlets, primarily driven by third-party referrals and nonsensitive news content. We discuss implications for online media control and media power.

If people enjoy giving, then why do they avoid fund-raisers? Partnering with the Salvation Army at Christmastime, we conducted a randomized field experiment placing bell ringers at one or both main entrances to a supermarket, making it easy or difficult to avoid the ask. Additionally, bell ringers either were silent or said “please give.” Making avoidance difficult increased both the rate of giving and donations. Paradoxically, the verbal ask dramatically increased giving but also led to dramatic avoidance. We argue that this illustrates sophisticated awareness of the empathy-altruism link: people avoid empathic stimulation to regulate their giving and guilt.

In an effort to bring greater efficiency, equity, and transparency to the criminal justice system, statistical risk assessment tools are increasingly used to inform bail, sentencing, and parole decisions. We examine New York City's stop-and-frisk program, and propose two new use cases for personalized risk assessments. First, we show that risk assessment tools can help police officers make considerably better real-time stop decisions. Second, we show that such tools can help audit past actions; in particular, we argue that a sizable fraction of police stops were conducted on the basis of little evidence, in possible violation of constitutional protections.

We propose a new framework for optimal experimentation, which we term the “A/B testing problem.” Our model departs from the existing literature by allowing for fat tails. Our key insight is that the optimal strategy depends on whether most gains accrue from typical innovations or from rare, unpredictable large successes. If the tails of the unobserved distribution of innovation quality are not too fat, the standard approach of using a few high-powered “big” experiments is optimal. However, if the distribution is very fat tailed, a “lean” strategy of trying more ideas, each with possibly smaller sample sizes, is preferred. Our theoretical results, along with an empirical analysis of Microsoft Bing’s EXP platform, suggest that simple changes to business practices could increase innovation productivity.