DP15217 Comparing Forecast Performance with State Dependence
|Author(s):||Florens Odendahl, Barbara Rossi, Tatevik Sekhposyan|
|Publication Date:||August 2020|
|Keyword(s):||Forecast evaluation, Pockets of Predictability, State Dependence|
|JEL(s):||C52, C53, G17|
|Programme Areas:||Monetary Economics and Fluctuations|
|Link to this Page:||cepr.org/active/publications/discussion_papers/dp.php?dpno=15217|
We propose a novel forecast comparison methodology to evaluate models' relative forecasting performance when the latter is a state-dependent function of economic variables. In our benchÂ¬mark case, the relative forecasting performance, measured by the forecast loss differential, is modeled via a threshold model. Importantly, we allow the threshold that triggers the switch from one state to the next to be unknown, leading to a non-standard test statistic due to the presence of a nuisance parameter. Existing tests either assume a constant out-of-sample forecast performance or use non-parametric techniques robust to time-variation; consequently, they may lack power against state-dependent predictability. Importantly, our approach is applicable to point forecasts as well as predictive densities. Monte Carlo results suggest that our proposed test statistics perform well in ï¬?nite samples and have better power than existing tests in selecting the best forecasting model in the presence of state dependence. Our test statistics uncover "pockets of predictability" in U.S. equity premia forecasts; the pockets are a state-dependent function of stock market volatility. Models using economic predictors perform signiï¬?cantly worse than a simple mean forecast in periods of high volatility, but, in periods of low volatility, the use of economic predictors may lead to small forecast improvements.