Data-driven lending relies on the calibration of models using training periods. We find that this type of lending is not resilient in the presence of economic conditions that are materially different from those experienced during the training period. Using data from a small business fintech lending platform, we document that the small business credit supply collapsed during the COVID-19 crisis of March 2020 even though the demand for loans doubled relative to pre-pandemic levels. As the month progressed, most lenders significantly reduced or halted their lending activities, likely due to the heightened risk of model miscalibration under the new economic conditions.