DP15217 Comparing Forecast Performance with State Dependence

Author(s): Florens Odendahl, Barbara Rossi, Tatevik Sekhposyan
Publication Date: August 2020
Keyword(s): Forecast evaluation, Pockets of Predictability, State Dependence
JEL(s): C52, C53, G17
Programme Areas: Monetary Economics and Fluctuations
Link to this Page: cepr.org/active/publications/discussion_papers/dp.php?dpno=15217

We propose a novel forecast comparison methodology to evaluate models' relative forecasting performance when the latter is a state-dependent function of economic variables. In our bench¬mark case, the relative forecasting performance, measured by the forecast loss differential, is modeled via a threshold model. Importantly, we allow the threshold that triggers the switch from one state to the next to be unknown, leading to a non-standard test statistic due to the presence of a nuisance parameter. Existing tests either assume a constant out-of-sample forecast performance or use non-parametric techniques robust to time-variation; consequently, they may lack power against state-dependent predictability. Importantly, our approach is applicable to point forecasts as well as predictive densities. Monte Carlo results suggest that our proposed test statistics perform well in �nite samples and have better power than existing tests in selecting the best forecasting model in the presence of state dependence. Our test statistics uncover "pockets of predictability" in U.S. equity premia forecasts; the pockets are a state-dependent function of stock market volatility. Models using economic predictors perform signi�cantly worse than a simple mean forecast in periods of high volatility, but, in periods of low volatility, the use of economic predictors may lead to small forecast improvements.