When the Swiss National Bank (SNB) announced on 15 January that it would no longer intervene to keep the franc/euro exchange rate at 1.20 (Swiss National Bank 2015), the franc appreciated by 16% that day – an almost unprecedented one-day movement for a major currency. Leaving aside the economic rationale for the announcement, what does its impact say about state-of-the-art models for risk forecasting?
To address this question, we downloaded the history of the daily franc/euro exchange rate from the ECB, and applied a range of risk forecast models and methods to returns on the exchange rate.1 The exchange rate can be seen in Figure 1.
The analysis presented here applies to currently used risk models as implemented by banks and approved and even mandated by the regulatory authorities. In other words, we focus our attention on models that might be favoured in the trading book regulations in Basel III (Basel Committee 2013). While one might argue that market participants would use more sophisticated models, regulated institutions increasingly have no other choice.
Figure 1. Swiss franc/euro exchange rate
Risk before and after the announcement
We applied our range of risk measures to returns on the euro/franc exchange rate, and report the outcomes in Table 1 below – both the risk on the day of the announcement and two business days after. We also report model risk as measured by the risk ratio methodology of Danielsson et al. (2014).
We find considerable disagreements among the various models before the event – for example, model risk is 8.8 using the value at risk measure and 13.4 using the expected shortfall measure. This is not surprising when we consider the evolution of the returns used in the estimation, as seen in Figure 2, and the fact that the various regimes in the data affect the risk measures very differently.
Table 1. Risk forecasts before and after the Swiss announcement
After the Swiss National Bank made its announcement, the franc appreciated by 16%, strongly affecting the risk forecasts on subsequent days. Table 1 shows the risk forecasts for two days after the announcement, where we find that every risk measure is affected – some very little and others considerably. The impact on the various risk measures is entirely consistent with what one might expect given the data and the announcement.
It seems clear from Table 1 that the risk models significantly under-appreciated the risk before the announcement. But it is equally worrying that they vastly overstate the risk after it. After all, the franc has just jumped from an artificial peg to what the market considers fair value. The observed jump to fair value tells us nothing about future risk because the jump was created by the gap between the pegged value and fair value, and that gap no longer exists.
Figure 2. Returns on euro/franc over the 1,000-day estimation window
How likely was the announcement based on the various methods?
We can also take the estimated risk models for the event day and calculate the implied probability of the event. We report that in Table 2 for all the methods except historical simulation, where it is not possible to do the calculation. By comparison, the age of the earth is estimated to be 5×109 years.
Table 2. Probability of the Swiss announcement as predicted by the risk models
This result makes it clear how badly standard risk models struggle in capturing tail risk, including those methods such as extreme value theory and t-GARCH specifically designed to achieve this. For completeness, we present several alternative ways of doing the analysis in the accompanying webpage, but the results do not change materially.For two of the methods, the probability was so low that it could not be calculated; for one method, moving average, it is extremely low; the Student-t GARCH thought that such an exchange rate move happened once every 2.1 million years; and only extreme value theory gave a somewhat reasonable number of once a century.
Perceived risk and actual risk
The fundamental problem is that the risk models considered here are inherently reactive as their main input is observed market data. When there are external factors dampening the movements of the data, such as a central bank capping the exchange rate, day-to-day movements become very small. Risk models observe this low volatility and deduce that there is low risk. Such models are essentially incapable of capturing the risk of structural breaks, or other tail events – perhaps with the exception of extreme value theory.
We have denoted such risk forecasts as perceived risk in Danielsson et al. (2012) – that is, the risk captured by backward-looking risk models. Perceived risk contrasts with actual risk, which is ever-present but very difficult to measure.
If one studies the history of the euro/franc exchange rate (especially the sharp movement in 2011) and also considers that the franc has only been holding at 1.20 to the euro because of increasingly costly central bank interventions, surely it was somewhat likely that the Swiss National Bank would abandon the peg. And in that case, it was inevitable that the exchange rate would jump considerably. Viewed in this light, the 16% move tells us almost nothing about the likely magnitude of future moves in the Swiss Franc, but something useful about the future of other currencies that are currently pegged.
If one were to apply basic economic analysis to the question of risk forecasting the exchange rate of the franc, it would seem sensible to consider other economic variables. The risk models considered here are simply not designed for such analysis and we would need different approaches, of which several possibilities exist.
That the industry standard and the regulator-approved risk models considered here cannot easily incorporate such analysis, or otherwise capture structural breaks, signals one of their greatest weaknesses.
The Swiss announcement on 15 January caught almost everybody by surprise. Such an event was certainly highly unlikely based on many standard risk forecast models, because they are based on the observation of day-to-day risk and are consequently unable to encompass the extreme events that arise when circumstances change in a fundamental way.
This event demonstrates a fundamental problem in the Basel III trading book regulations, since they are set on capturing market risk with exactly the methodologies discussed here. There is no reason to believe that the Basel methodologies will perform any better in predicting the risk of future events.
Macro-prudential regulators sometimes also rely on market data-based risk measures that depend on the methods discussed here, so they should also be concerned. The current crop of systematic risk forecast methods – including conditional value at risk, systemic expected shortfall, marginal expected shortfall, and the SRISK systemic risk measure – are subject to exactly the same results as reported here, and inherit the tendency to fail when extreme events occur. It seems more fruitful to move away from such market risk- and market data-based systemic risk analysis to more fundamental approaches, such as the financial cycles of Borio (2012).
Perhaps the lessons from the Swiss announcement will alert us to the problem of relying too much on current risk forecast methodologies before it's too late.
Basel Committee on Banking Supervision (2013), “Fundamental review of the trading book: A revised market risk framework”, Technical report.
Borio, Claudio (2012), “The financial cycle and macroeconomics: What have we learnt?”, Bank for International Settlements Working Paper 395.
Danielsson, J, K James, M Valenzuela, and I Zer (2014), “Model risk and the implications for risk management, macroprudential policy, and financial regulations”, VoxEU.org, 8 June.
Danielsson, J, J-P Zigrand, and H S Shin (2012), “Endogenous Extreme Events and the Dual Role of Prices”, Annual Reviews in Economics, Volume 4 on the Economics of Extreme Events.
Swiss National Bank (2015), “Swiss National Bank discontinues minimum exchange rate and lowers interest rate to -0.75%”, Press release, 15 January.
1 The risk forecast methods are historical simulation (HS), moving average (MA), exponentially weighted moving average (EWMA), GARCH, Student-t GARCH, and extreme value theory (EVT). The probability is 1%, the estimation window is 1,000 days, and the risk measures are value at risk (VaR) and expected shortfall (ES). The risk is based on a portfolio of €1,000. For technical details, see the accompanying webpage at http://www.modelsandrisk.org/Swiss.