The instability of leading indicators in forecasting Austrian inflation: lessons from the COVID-19 pandemic and the energy crisis (OeNB Bulletin Q4/24)

Friedrich Fritzer ¹

JEL classification: C53, E37, C50
Keywords: macroeconomic forecasting, inflation, leading indicators, forecast combination

Four times a year (in March, June, September and December), the Oesterreichische Nationalbank (OeNB) produces its Economic Outlook for Austria, aimed at guiding policymaking in general and at assessing price stability in particular. With respect to inflation forecasting, these projections employ time series models along the lines of Fritzer, Moser and Scharler (2002) as well as semistructural models. In the time between projection rounds, new economic indicators are published continuously. For forecasting inflation and evaluating current price stability alike, it is important to have a solid basis for assessing the information content indicators have for these purposes. One way to do this is to apply autoregressive distributed lag models in which inflation is explained by its own lags and the lags of individual indicators. Influential studies on the use of univariate leading indicators in forecasting inflation were carried out for the United States and the euro area by Cecchetti et al. (2000), Banerjee and Marcellino (2006) and Banerjee et al. (2005). The present study is based on these papers and tries to examine leading indicator properties for predicting Austrian inflation. ² Furthermore, we assess the robustness of these indicators over time and explore the question whether there are reliable indicators that could predict the rapid surge of inflation in the period after 2021 and its downturn from the beginning of 2023.

Indicators frequently send diverse signals in terms of the strength and direction of inflation prospects. This makes it complicated to assess whether inflation is about to go up, fall or stabilize. Therefore, it might prove advantageous to condense the information content of a number of indicators into an aggregate (or composite indicator). Bates and Granger (1969) were perhaps the most influential researchers to spread the idea of combining leading indicator forecasts. There are different ways to do this.

• One – theoretically ideal – way would be to weight indicators according to their forecasting errors or, more specifically, to weight them proportionally to the inverse of their forecasting error variance, an approach which would downweigh indicators with large forecasting errors.

• A much simpler weighting scheme would be to attach equal weights to leading indicator forecasts. Surprisingly, some researchers found that the simple average of leading indicator forecasts proved to be more successful than more sophisticated weighting schemes (see e.g. Stock and Watson, 2004). This phenomenon has been described as the “forecast combination puzzle.”

• Another way to summarize the information content of many indicators would be to first condense the information they contain into a few factors and then use these factors to forecast inflation. Stock and Watson (1999), for instance, used such factor models to forecast US inflation.

This paper also investigates these three composite indicators with respect to their accuracy in forecasting inflation in Austria. It is structured as follows: In the first section, we present the data used. In section 2, we describe inflation forecasts based on individual indicators in terms of their construction. Furthermore, we assess indicators’ forecasting accuracy. Section 3 deals with the construction of composite leading indicators and their forecasting quality. Section 4 concludes.

1 Data

In total, we investigate 25 macroeconomic indicators (see table 1) that are forecast in the OeNB’s regular macroeconomic projection exercises. Therefore, they provide a solid basis for a consistent inflation forecasting framework, as we will show in section 2. These indicators comprise price and cost indicators (e.g. oil prices, domestic output prices for consumer goods, compensation per employee), indicators of macroeconomic activity (e.g. real GDP, the output gap and the unemployment gap as measures of the degree of capacity utilization) and financial market indicators. The link between indicators of macroeconomic activity and inflation is likely to be tight in cases in which these variables are good measures of capacity utilization and in which inflation is determined from the demand side. Inflation tends to rise with the degree of utilization of production factors. Financial indicators are potentially useful predictors of inflation as they are inherently forward looking and reflect the transmission of monetary policy (e.g. interest rate changes).

We use quarterly data from Q1 99 to Q4 23. Most of the data used in this empirical analysis are seasonally adjusted, except for oil prices, USD-EUR exchange rates, interest rates, domestic output prices and the inflation rate as measured by the Harmonised Index of Consumer Prices (HICP), as there is no or little seasonality in these variables. However, the modeling approach explained in section 2 effectively accounts for seasonality as well. In this empirical analysis, the variables interest rates, output gap, profit margin, unemployment rate, non-accelerating inflation rate of unemployment (NAIRU), i.e. the lowest unemployment rate consistent with stable inflation in the long run, and the unemployment gap are used in levels. All other variables are transformed into annual percentage changes.

2 Inflation forecasts based on individual indicators

To measure the informational content of individual indicators, we estimated an empirical specification to best represent the relationship between HICP inflation and each individual inflation indicator.

More specifically, we estimated an autoregressive distributed lag (ARDL) model of the form

\(\pi_{t} = \alpha + \sum_{k = 1}^{4}{\beta_{k}\pi_{t - k}} + \sum_{q = 0}^{4}{\gamma_{q}x_{t - q}} + \varepsilon_{t},\) (1),

using the ordinary least squares method, where \(\pi\) is the annual inflation rate, \(x\) is the inflation indicator, \(\varepsilon\) an independent and identically distributed (IDD) disturbance term, t the time subscript and t–k and t–q are the k -times and q -times lagged value of the variables. ³ In a first step, the maximum lag lengths for inflation and the inflation indicators was restricted to be four, as we deal with quarterly data. In a second step, we chose the optimal lag specification of inflation and the inflation indicators to minimize the Akaike information criterium. ⁴ Lagged values of the annual inflation rate account for serial correlation in the variable, while lagged values of the indicators measure information content with respect to explaining inflation. As a benchmark, we also use an autoregression that only uses past inflation to forecast inflation. ⁵ For an indicator to be useful in inflation forecasting, its ARDL forecast must beat the autoregressive forecast. To conduct inflation forecasts based on indicators, we must have an estimate of how these indicators will develop over the forecasting period. We use the realized values of these indicators. This is certainly a strong assumption, however – an assumption that allows us to isolate the information content of the indicators with respect to forecasting inflation. The results of our forecast evaluation exercise must be interpreted against this background, as some indicators are inherently difficult to forecast (e.g. USD-EUR exchange rates and oil prices) or can be subject to significant data revisions. In our forecasting practice, however, we would use projected variables derived from the OeNB’s macroeconometric model of the Austrian economy.

This framework is used to forecast Austrian inflation one to eight quarters ahead in the following way: Our forecast evaluation exercise involves non-overlapping estimation and forecasting periods such that only data from before the forecasting date are used in the estimation to produce out-of-sample forecasts. More specifically, in a first step, the estimation equation is determined for the period from Q1 99 to Q1 07, followed by a forecast of the HICP inflation rate up to eight quarters ahead (from Q2 07 to Q1 09). Subsequently, the relationship between the indicator used and HICP inflation is estimated for the first period plus one quarter and an inflation forecast is conducted for the period up to eight quarters ahead. This procedure is repeated until the entire forecast evaluation period (Q2 07 to Q4 23) is covered. It results in 67 one-quarter-ahead forecasts, 64 four-quarters-ahead forecasts and 60 eight-quarters-ahead forecasts, which are then compared with the realized values. Forecasting accuracy is measured using the root mean squared prediction error (RMSPE), which assesses the average deviation of the forecast from the actual outcome. ⁶

Table 2.1 shows the results of the described evaluation process for three different forecasting horizons within the period from Q2 07 to Q4 19, i.e. without taking into account the COVID-19 shock and the energy price shock. Below, we investigate whether the results remain robust when these shocks are included in the evaluation period. Table 2.1 also gives the rank of the indicators according to the RMSPEs. These rankings must be interpreted against the background that RMSPEs occasionally differ only slightly from each other. In this analysis, we did not test whether there were any significant differences between RMSPEs. For the RMSPEs themselves, see tables A1.1 and A1.2 in the annex. Table 2.1 reports the forecasting performance of the examined indicators until end-2019.

For that period, domestic output prices for consumer goods are found to perform best, meaning that across all forecasting horizons, this inflation indicator provides the most accurate forecasts of the HICP inflation rate. Indicators that also show good forecasting qualities across all forecasting horizons include the output gap and foreign demand. ⁷ A partially good forecasting performance can be reported for the lending rate to businesses (one and four quarters ahead), competitors’ import prices and total employment (eight quarters ahead) as well as oil prices (one quarter ahead). The euro-denominated oil price, for instance, is found to be among the best inflation indicators, but its forecasting performance is very good only for one-quarter-ahead forecasts and deteriorates significantly with the length of the forecasting horizon. For eight quarters ahead, it is even among the five worst inflation indicators examined. Conversely, competitors’ import prices and total employment show good leading indicator properties over longer forecasting horizons (eight quarters ahead) but not for the shorter horizons of one quarter ahead and four quarters ahead. However, we must take into account that indicator performance is measured as an average across the entire forecasting period. This masks the possibility that large episodic errors might be mitigated over time. For instance, the output gap and domestic output prices for consumer goods considerably overestimated inflation developments in 2016 and 2017, as inflation during this period was largely driven by oil prices. Circumstances like these might call for a combination of indicators, which we investigate in section 3. When considering an autoregressive model of inflation as a benchmark, we find that 9 to 13 indicators beat autoregressions over the one- to eight-quarters-ahead forecasting horizon (see table A2.1 in the annex).

As a robustness check, we compared these results with the evaluation carried out over the period including the COVID-19 and energy crises. Extending the evaluation period until the end of Q4 23 changes the forecasting performance of indicators to some extent (see table 2.2).

Comparing RMSPEs for the entire evaluation period (until Q4 23) with those for the period until Q4 19, shows that their levels increase substantially for almost all indicators (compare tables A1.1 and A1.2 in the annex). Part of this increase is attributable to the fiscal measures that were introduced in Austria in 2022 and 2023 to dampen energy price inflation ⁸ , given that these measures are not accounted for in our forecast evaluation exercise. According to our estimates, the impact of the fiscal measures on Austria’s HICP inflation amounted to –0.4 percentage points in 2022 and –0.9 percentage points in 2023.

Competitors’ import prices are the best indicator over the entire forecasting period. However, oil prices and domestic output prices of consumer goods perform well for one, four and eight quarters ahead too. Compared with the period until Q4 19, the forecasting performance of competitors’ import prices and of oil prices improved. In the period until Q4 19, oil prices only performed well for one-quarter-ahead forecasts and competitors’ import prices did so only for four- and eight-quarters-ahead forecasts. Domestic output prices for consumer goods maintained their good forecasting performance but fell behind competitors’ import prices in particular when measured in terms of RMSPEs (see tables A1.1 and A1.2). The output gap lost its good leading indicator properties for the short term (one quarter ahead) forecast when compared to the evaluation for the period until Q4 19 but maintained relatively good forecasting properties for the medium term forecast (four quarters ahead). However, we must not forget that extending the evaluation period from Q4 19 to Q4 23 implies that only few observations are available for the four-quarters-ahead horizon (13) and even less for the eight-quarters-ahead horizon (10).

When comparing the performance of individual indicators with the benchmark of an autoregressive process that only uses past inflation to forecast inflation, we find that the longer the forecasting horizon the better the forecasting accuracy of the individual indicators as long as the entire period until end-2023 was evaluated (see tables A1.1 and A1.2). This was not the case for the shorter evaluation period until end-2019. Hence, we can conclude that during the COVID-19 pandemic and the energy crisis, the autoregressive benchmark deteriorated in comparison to the forecasting performance of the individual indicators. This may partly be due to our forecast evaluation framework, which uses realized values for the individual indicators. In times of shocks, when indicators change more strongly than in normal times, advance information on the development of variables becomes more important. Autoregressive processes, by contrast, use only past information. This might be sufficient for inflation forecasting in normal times but less so in turbulent times.

3 Inflation forecasts based on composite indicators

Building on our evaluation of individual indicators, we explore the question whether their information content can be combined into composite indicators that are superior to individual indicators. Forecasts based on indicator combinations might outperform individual indicator forecasts as they pool the information contained in a number of indicators. Furthermore, Clements and Hendry (2002) showed that structural breaks might be tackled by way of combining indicators, as breaks may affect specific indicators more than others. For instance, the problems with measuring the output gap during the COVID-19 pandemic were not mirrored to the same extent in the development of domestic output prices for consumer goods. In this forecast evaluation exercise, we test three commonly used composite indicators: (1) the simple average of all leading indicator forecasts; (2) the optimal weighting of individual indicator forecasts based on their prediction errors according to Bates and Granger (1969). More specifically, this means that indicator \(i\) (one of the indicators in table 1) is weighted proportionally to the inverse of its squared prediction error \(\sigma_{i}^{2}\ \), i.e.

\(w_{i} = \frac{1/\sigma_{i}^{2}}{\sum_{i}^{}\frac{1}{\sigma_{i}^{2}}}\) (2).

In our empirical investigation, the forecast evaluation is conducted in a recursive manner (see section 2), constantly updating the weights; and (3) the aggregation of indicators using factor analysis. This is a statistical technique that condenses a large number of indicators into one or a few summary measures. The first summary measure (the so-called first factor) is defined as the linear combination of the indicators which explains most of the variability of the indicators. We therefore choose this first summary measure in our forecasting exercise. This technique is applied i.a. by Stock and Watson (1999). The first two composite indicators listed above combine inflation forecasts while the third combines indicators into a summary measure which is subsequently used in forecasting inflation. Our selection of these combination methods is by no means arbitrary. The simple average deserves attention as it sometimes proves superior to other aggregates in empirical studies. In a recent survey on forecast combinations, Wang et al. (2023) stated, “Despite the explosion of various popular and sophisticated combination methods, empirical evidence and extensive simulations repeatedly show that the simple average with equal weights often outperforms more complicated weighting schemes.” From a theoretical point of view, forecast combinations based on optimal weights according to Bates and Granger (1969) are, however, superior to the simple average with equal weights, and forecasts using factor models are an acknowledged method for working with many variables.

Table 3.1 shows the forecasting performance of the three composite indicators in question as well as the forecasting errors of some of the best individual indicators for the evaluation period from Q2 07 to Q4 19. The volatile period driven by the COVID-19 and energy crises is evaluated in a subsequent step.

During the period until the end of 2019, the first factor extracted from the 25 variables specified in table 1 is the best-performing composite indicator. It puts most of the weight (referred to as “loading”) on oil prices, followed by a wide margin by the unemployment rate and foreign demand. Next in line with roughly equal weights are consumer loans, competitors’ import prices, loans to consumers and businesses, real investment and real GDP. ⁹ We find that across all forecasting horizons, the first factor scheme consistently outperforms both the simple average scheme and the composite indicator scheme according to Bates and Granger (1969). Bates and Granger weights are equal to the simple arithmetic mean for one-quarter-ahead forecasts but much better for longer forecasting horizons. Therefore, the so-called forecast combination puzzle is not supported for this period. Additionally, we find that the best individual indicators lag significantly behind the first component derived from the factor analysis for the medium- and long-term forecasting horizon (four quarters ahead and eight quarters ahead). Comparing the results of the equal weighting scheme and the optimal weighting scheme with the best individual indicator forecasts shows that the weighting combination schemes prove to be broadly equal for the short-term horizon of one quarter ahead while over the long-term horizon of eight quarters ahead, they tend to be slightly better.

Table 3.2 reports the results we obtained when including the COVID-19 pandemic and the energy crisis in our evaluation of indicators’ forecasting performance.

When we compare the size of the forecasting errors reported in table 3.2 with those for the period until end-2019, we find that the forecasting performance of individual indicators and combined indicators worsens.

For four-quarters-ahead and eight-quarters-ahead forecasts, individual indicators surpassed the forecasting performance of the first factor extracted using factor analysis, while the first factor performs best only over the short run (one quarter ahead). Over the entire forecasting horizon, the weighting combination schemes yielded less accurate results than the two best individual indicators (competitors’ import prices and domestic output prices for consumer goods). So in this case, the combined indicators did not prove to be a hedge against structural breaks. However, the good results in terms of forecasting accuracy for individual indicators that we recorded over the entire evaluation period until end-2023 must be interpreted against the background of the large size of the forecasting errors. The failure of the composite indicator forecasts to outperform individual indicator forecasts in an unstable environment calls for an investigation into the reasons, which might help improve the forecasting performance of composite indicators. One reason could be that the accuracy of individual indicator forecasts changes over time. In that case, the weights of indicators with a deteriorating forecasting accuracy will need to be downgraded.

4 Conclusions

In this analysis, 25 macroeconomic indicators were tested for their leading indicator properties concerning the Austrian HICP inflation rate. Additionally, we evaluated three methods that combine the information content of these indicators with the purpose of forecasting inflation. One major result is that leading indicator properties change over time. For instance, international commodity prices (oil prices, import prices) became more important as leading indicators after Q1 00, while economic activity variables (output gap, unemployment gap) lost their leading indicator properties with respect to HICP inflation. Structural changes during the COVID-19 and energy price shocks seem to be key events that changed indicators’ forecasting performance. An update of indicators in use in inflation forecasting practice therefore seems to be warranted, in particular when it comes to accounting for potential economic shocks. During economically more tranquil periods, we could continue to rely on established indicators.

Overall, our analysis might provide useful inputs to monitoring inflation developments. In particular, the forecasting framework using individual and composite indicators may generate information complementary to that produced by other models and may thus prove helpful in projection exercises.

References

Bates, J. M. and C. W. J. Granger. 1969. The Combination of Forecasts. In: Operational Research Society 20(4). 451–468.

Banerjee, A. and M. Marcellino. 2006. Are there any reliable leading indicators for the US inflation and GDP growth? In: International Journal of Forecasting 22. 137–151

Banerjee, A., M. Marcellino and I. Masten. 2005. Leading indicators for Euro area inflation and GDP growth. In: Oxford Bulletin of Economics and Statistics 67. 785–813.

Cecchetti, S. G., R. S. Chu and C. Steindel. 2000. The Unreliability of Inflation Indicators. In: Current Issues in Economics and Finance 4(6). Federal Reserve Bank of New York.

Clements, M. B. and D. F. Hendry. 2002. Pooling forecasts. In: Econometrics Journal 5. 1–26.

Marcellino, M. 2006. Leading Indicators. In: G. Elliott, C.W.J. Granger and A. Timmermann (eds), Handbook of Economic Forecasting 1. Amsterdam: Elsevier.

Fritzer F., G. Moser and J. Scharler. 2002. Forecasting Austrian HICP and its Components using VAR and ARIMA models. In: OeNB Working Paper 73.

Stock, J. H. and M. W. Watson. 1999. A comparison of linear and non-linear univariate models for forecasting macroeconomic time series. In: R. Engle and H. White (eds), Cointegration, causality, and forecasting: A festschrift in honor of Clive W.J. Granger. Oxford University Press. 1–44.

Stock, J. H. and M. W. Watson. 2004. Combination Forecasts of Output Growth in a Seven-Country Data Set. In: Journal of Forecasting 23. 405–430.

Stock, J. H. and M. W. Watson. 2016. Factor models and structural vector autoregressions in macroeconomics. In: J.B. Taylor and H. Uhlig (eds). Handbook of Macroeconomics. Amsterdam. Elsevier. 415–525.

Wang, X., R. J. Hyndman, F. Li and Y. Kang. 2023. Forecast combinations: An over 50-year review. In: International Journal of Forecasting 39. 1518–1547.

Annex