Historical Simulation (HS) and its variant, the Filtered Historical Simulation (FHS), are the most popular Value-at-Risk forecast methods at commercial banks. These forecast methods are traditionally evaluated by means of the unconditional backtest. This paper formally shows that the unconditional backtest is always inconsistent for backtesting HS and FHS models, with a power function that can be even smaller than the nominal level in large samples. Our findings have fundamental implications in the determination of market risk capital requirements, and also explain Monte Carlo and empirical findings in previous studies. We also propose a data-driven weighted backtest with good power properties to evaluate HS and FHS forecasts. A Monte Carlo study and an empirical application with three US stocks confirm our theoretical findings. The empirical application shows that multiplication factors computed under the current regulatory framework are downward biased, as they inherit the inconsistency of the unconditional backtest.

The Basel Committee and the Financial Stability Board require a consensus on the identification of characteristics that make a financial institution more prone than others to be severely hit by systemic shocks. This paper introduces a new tool to achieve this goal: a model for the Conditional Average Systemic Effects (CASE). The CASE quantifies the average effect of a system wide shock or market downturn on the profit and loss account of a bank, a firm or on the return of an asset. We propose a linear model for CASE with heterogeneous effects in observable characteristics. These models complement alternative measures of systemic risk and allow researchers to identify the determinants of the vulnerability of a given financial institution. We develop bootstrap inference that accounts for both estimation risk and model misspecification risk, and show the utility of our results in Monte Carlo simulations and an empirical application to 100 large U.S. financial firms.