A main focus in economics is how to design optimal policies in second-best situations, which often requires a trade-off between giving some individuals more than they deserve, false positives, and others less than they deserve, false negatives. This paper provides novel evidence on people's second-best fairness preferences from large-scale experimental studies in the United States and Norway. The majority of people are more concerned with false negatives than with false positives, but we document substantial heterogeneity in second-best fairness preferences between the countries and across the political spectrum. The findings shed light on the political economy of social insurance and redistribution.