Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The expost threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. Given probabilistic model output, it is intuitive that a decision rule is independent of the data or model specification, as thresholds on probabilities represent a willingness to issue a false alarm vis-à-vis missing a crisis. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Artikel Lesen
Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?
Peter Sarlin, Gregor von Schweinitz
Abstract
Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The ex-post threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance. We propose two alternatives for threshold setting: (i) including preferences in the estimation itself and (ii) setting thresholds ex-ante according to preferences only. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.
Artikel Lesen
The Impact of Preferences on Early Warning Systems - The Case of the European Commission's Scoreboard
Tobias Knedlik
European Journal of Political Economy,
2014
Abstract
The European Commission’s Scoreboard of Macroeconomic Imbalances is a rare case of a publicly released early warning system. It allows the preferences of the politicians involved to be analysed with regard to the two potential errors of an early warning system – missing a crisis and issuing a false alarm. These preferences might differ with the institutional setting. Such an analysis is done for the first time in this article for early warning systems in general by using a standard signals approach, including a preference-based optimisation approach, to set thresholds. It is shown that, in general, the thresholds of the Commission’s Scoreboard are set low (resulting in more alarm signals), as compared to a neutral stand. Based on political economy considerations the result could have been expected.
Artikel Lesen
Outperforming IMF Forecasts by the Use of Leading Indicators
Katja Drechsel, Sebastian Giesen, Axel Lindner
IWH Discussion Papers,
Nr. 4,
2014
Abstract
This study analyzes the performance of the IMF World Economic Outlook forecasts for world output and the aggregates of both the advanced economies and the emerging and developing economies. With a focus on the forecast for the current and the next year, we examine whether IMF forecasts can be improved by using leading indicators with monthly updates. Using a real-time dataset for GDP and for the indicators we find that some simple single-indicator forecasts on the basis of data that are available at higher frequency can significantly outperform the IMF forecasts if the publication of the Outlook is only a few months old.
Artikel Lesen
Predicting Financial Crises: The (Statistical) Significance of the Signals Approach
Makram El-Shagi, Tobias Knedlik, Gregor von Schweinitz
Abstract
The signals approach as an early warning system has been fairly successful in detecting crises, but it has so far failed to gain popularity in the scientific community because it does not distinguish between randomly achieved in-sample fit and true predictive power. To overcome this obstacle, we test the null hypothesis of no correlation between indicators and crisis probability in three applications of the signals approach to different crisis types. To that end, we propose bootstraps specifically tailored to the characteristics of the respective datasets. We find (1) that previous applications of the signals approach yield economically meaningful and statistically significant results and (2) that composite
indicators aggregating information contained in individual indicators add value to the signals approach, even where most individual indicators are not statistically significant on their own.
Artikel Lesen
Zur Aussagekraft von Frühindikatoren für Staatsschuldkrisen in Europa
Tobias Knedlik, Gregor von Schweinitz
Wirtschaft im Wandel,
Nr. 10,
2011
Abstract
Die Schulden- und Vertrauenskrise in Europa hat eine intensive Diskussion über die makroökonomische Koordinierung ausgelöst. Die bestehenden Institutionen, darunter auch der Stabilitäts- und Wachstumspakt, haben sich als Krisenpräventions- und Krisenmanagementinstrumente nicht bewährt. Ein Vorschlag in der gegenwärtigen Debatte lautet, anhand geeigneter Frühindikatoren eine regelmäßige und systematische makroökonomische
Überwachung vorzunehmen, um sich anbahnende Krisen früh erkennen und darauf reagieren zu können. Dieser Beitrag stellt die Prognosegüte von vier vorgeschlagenen Indikatorensets vergleichend dar, wobei sowohl die Güte
von Einzelindikatoren als auch die Güte aggregierter Gesamtindikatoren betrachtet werden. Die verschiedenen Einzelindikatoren weisen eine sehr unterschiedliche Prognosequalität auf, wobei sich neben dem Staatsdefizit
besonders die Arbeitsmarktindikatoren, die private Verschuldung und der Leistungsbilanzsaldo durch eine hohe Prognosegüte auszeichnen. Unter den Gesamtindikatoren schneiden besonders jene gut ab, die sowohl viele unterschiedliche als auch besonders gute Einzelindikatoren beinhalten. Deshalb wird für den Einsatz eines breit basierten Gesamtindikators bei der makroökonomischen Überwachung plädiert. Dieser sollte zudem aus gleichgewichteten Einzelindikatoren zusammengesetzt sein, um der Tatsache Rechnung zu tragen, dass die Ursachen künftiger Krisen vorab nicht bekannt sind.
Artikel Lesen
The Financial Crisis from a Forecaster’s Perspective
Katja Drechsel, Rolf Scheufele
Abstract
This paper analyses the recession in 2008/2009 in Germany, which is very different from previous recessions, in particular regarding its cause and magnitude. We show to what extent forecasters and forecasts based on leading indicators fail to detect the timing and the magnitude of the recession. This study shows that large forecast errors for both expert forecasts and forecasts based on leading indicators resulted during this recession which implies that the recession was very difficult to forecast. However, some leading indicators (survey data, risk spreads, stock prices) have indicated an economic downturn and hence, beat univariate time series models. Although the combination of individual forecasts provides an improvement compared to the benchmark model, the combined forecasts are worse than several individual models. A comparison of expert forecasts with the best forecasts based on leading indicators shows only minor deviations. Overall, the range for an improvement of expert forecasts during the crisis compared to indicator forecasts is relatively small.
Artikel Lesen
Should We Trust in Leading Indicators? Evidence from the Recent Recession
Katja Drechsel, Rolf Scheufele
Abstract
The paper analyzes leading indicators for GDP and industrial production in Germany. We focus on the performance of single and pooled leading indicators during the pre-crisis and crisis period using various weighting schemes. Pairwise and joint significant tests are used to evaluate single indicator as well as forecast combination methods. In addition, we use an end-of-sample instability test to investigate the stability of forecasting models during the recent financial crisis. We find in general that only a small number of single indicator models were performing well before the crisis. Pooling can substantially increase the reliability of leading indicator forecasts. During the crisis the relative performance of many leading indicator models increased. At short horizons, survey indicators perform best, while at longer horizons financial indicators, such as term spreads and risk spreads, improve relative to the benchmark.
Artikel Lesen
The ADR Shadow Exchange Rate as an Early Warning Indicator for Currency Crises
Stefan Eichler, Alexander Karmann, Dominik Maltritz
Journal of Banking and Finance,
Nr. 11,
2009
Abstract
We develop an indicator for currency crisis risk using price spreads between American Depositary Receipts (ADRs) and their underlyings. This risk measure represents the mean exchange rate ADR investors expect after a potential currency crisis or realignment. It makes crisis prediction possible on a daily basis as depreciation expectations are reflected in ADR market prices. Using daily data, we analyze the impact of several risk drivers related to standard currency crisis theories and find that ADR investors perceive higher currency crisis risk when export commodity prices fall, trading partners’ currencies depreciate, sovereign yield spreads increase, or interest rate spreads widen.
Artikel Lesen
