For social scientists, developing an empirical connection between the physical appearance of a city and the behavior and health of its inhabitants has proved challenging due to a lack of data on urban appearance. Can we use computers to quantify urban appearance from street-level imagery? We describe Streetscore: a computer vision algorithm that measures the perceived safety of streetscapes. Using Streetscore to evaluate 19 American cities, we find that the average perceived safety has a strong positive correlation with population density and household income; and the variation in perceived safety has a strong positive correlation with income inequality.