What are risk models good for?
VoxEU Blog/Review Macroeconomic policy Financial Markets

What are risk models good for?

Risk models are easy targets for criticism, given their poor accuracy. So what are they good for? In this post, Jon Danielsson discusses four categories of risk model and how they might be used in real-world applications.

First posted on: 

modelsandrisk.org

 

One can endlessly criticise risk models, but that is just too nihilistic. So, what are they good for? There are three camps, the model believers, the rejectionists, and the healthy sceptics. I’m going to make the case for the last below.

It is easy to criticise risk models, depressingly easy. I certainly have been guilty of that like many others.

However, considering their accuracy, how should they be used in real-world applications?

Let’s classify them into four categories:

  1. Managing day-to-day risk
  2. Internal risk capital allocation
  3. Microprudential regulations
  4. Macroprudential regulations

Managing day-to-day risk

Risk models were really designed for managing day-to-day risk, say, on the trading floor.

Suppose you have Ann, Bill, and Jo trading the same stuff, and all of a sudden, Ann’s value at risk (VaR) relative to Bill’s and Jo’s, shoots up. Well then, you know that something is afoot. Perhaps Ann is taking too much risk or Bill and Jo too little, or perhaps the model messed up.

Regardless, it is a useful signal, and the intelligent risk manager will treated it that way. Of course, if the risk manager is the tick-the-box type (to be discussed later) then all bets are off, but let’s discard that.

Internal risk capital allocation

What this means is how a financial institution allocates capital between asset classes: does it want to decrease its exposure to European small caps and increase its holding of US junk bonds? Then the question is about the allocation of capital meant to be used for risk taking.

Here, the risk models become much more useless, well-illustrated by the Swiss FX shock. The risk model gives you a signal about day-to-day risk but misses the big events caused by other factors, such as the macroeconomy, the financial system, and the like.

Of course, the risk models can help, but should only be one of the many other decision factors.

So, to conclude, risk models are mostly useless for internal risk capital allocation.

Microprudential regulations

Micro is a broad church, and risk modelling has little to say about its bread-and-butter issues such as conduct.

And, even if much of micro is about risk, it is not of the statistical modelling variety. OK, before anyone objects, not of the VaR and expected shortfall (ES) type discussed here.

Statistical risk modelling can certainly be of help, if only to ensure that financial institutions are properly managing day-to-day risk.

The danger is if the micro regulators start relying on statistical risk models as a signal of the health of the financial institution, or some such.

Also, regulations have a tendency to degenerate into the tick-the-box type, and statistical risk modelling is particularly subject to that.

So overall, risk models can be helpful for micro, but can also be useless and even dangerous if used incorrectly.

Macroprudential regulations

Here, it is all about extreme outcomes, extreme tail risk and systemic risk. Statistical risk models have little or nothing to say about such risk, as discussed here.

Statistical risk models can be outright dangerous for macroprudential policies, or ‘macropru’. If the authorities formulate policy and attempt to control the system based on such a highly inaccurate signal, they may amplify risk when they should reduce it, or curtail risk when risk is needed.

The cost of type I and type II error is significant.

That means that systematic risk forecast methods such as SES, MES, CoVaR, SRISK, Sharpley, CISS, and the like, not only do not capture the risk they purport to do, but even worse, their usage increases systemic risk.

I discussed the reliability of such systematic risk measures in two papers: “Can we prove a bank guilty of creating systemic risk? A minority report” and “Model risk of risk models”.

I think a lot of policymakers know this. Fortunately, research based on applying standard market risk methodologies to forecasting systemic risk is not taken too seriously these days.