PROJECT A2
OECD FORECASTS
DURING & AFTER THE
FINANCIAL CRISIS
NAEC Seminar, 20 November 2013
Speakers: • Sveinbjorn Blondal, ECO, OECD • Nigel Pain, ECO, OECD • Christine Lewis, ECO, OECD
A POST-MORTEM
Motivation and context
• NAEC: reviewing the crisis and identifying lessons for future forecasting performance.
• Regular forecast post-mortems by ECO.
• Three aspects of the project:
– Quantitative evaluation of the forecasts.
– What structural, financial and policy-related factors are related to the errors?
– Interviews on post-crisis changes with experts from other international organisations.
Ov
erv
iew
2
• Three sets of forecasts:
– May Economic Outlook for the next year;
– November Economic Outlook for the next year;
– May Economic Outlook for the same year
• Outturn data
– The published data in May after the year being
forecast
• Focus on calendar year GDP growth forecasts
– The project also considered inflation forecasts
and quarterly forecasts for the large economies
3
Data set and definitions T
he
da
ta
PROPERTIES OF THE RECENT FORECAST
ERRORS
4
The downturn was not foreseen
• Forecasts of (Q4/Q4) OECD GDP growth in 2008 and 2009 were revised down substantially.
Pro
per
ties
of
the
fore
cast
err
ors
5
-3
-2
-1
0
1
2
3
May-07 Nov-07 May-08 Nov-08 May-09
Forecasts of GDP growth in 2008
Outturn
%
-3
-2
-1
0
1
2
3
May-08 Nov-08 May-09 Nov-09 May-10
Forecasts of GDP growth in 2009
Outturn
%
The recovery has been mixed
• OECD GDP growth (Q4/Q4) rebounded more quickly than initially expected in 2010.
• But disappointments resumed in 2011 and 2012
Pro
per
ties
of
the
fore
cast
err
ors
6
0
1
2
3
4
May-09 Nov-09 May-10 Nov-10 May-11
Forecasts of GDP growth in 2010
Outturn
%
0
1
2
3
4
May-10 Nov-10 May-11 Nov-11 May-12
Forecasts of GDP growth in 2011
Outturn
%
The biggest error in the 18-month ahead
calendar year growth forecasts was 2009.
Pro
per
ties
of
the
fore
cast
err
ors
7
Errors are typically smaller at shorter
forecast horizons.
Pro
per
ties
of
the
fore
cast
err
ors
Forecast error = outturn less forecast 8
On average, the errors for the BRIICS are
similar to those for OECD countries.
Pro
per
ties
of
the
fore
cast
err
ors
Forecast error = outturn less forecast 9
Pro
per
ties
of
the
fore
cast
err
ors
Forecast errors were largest in the
vulnerable euro area countries
10
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
OECD Euro areacore
Euro areavulnerable
Other OECDEurope
Rest ofOECD
BRIICS
Average growth forecast error 2007-12
May current year
Nov forecast for following year
May forecast for following year
%pt
Pro
per
ties
of
the
fore
cast
err
ors
The pattern of forecast accuracy is
similar, though differences are smaller
11
0
1
2
3
4
5
OECD Euro areacore
Euro areavulnerable
Other OECDEurope
Rest ofOECD
BRIICS
RMSE of GDP growth forecasts 2007-12
May current year
Nov forecast for following year
May forecast for following year
%pt
The root mean squared error (RMSE) could be used to put
confidence intervals around a point forecast.
EXPLAINING FORECAST ERRORS
12
Global interconnectedness rose substantially
prior to the crisis, with rising global & national
imbalances.
Ex
pla
inin
g t
he
erro
rs
13
0
40
80
120
160
0
20
40
60
80
1995 2007 1995 2007 1995 2007
Trade openness(LHS, % of GDP)
Financial openness(RHS, % of GDP)
Foreign banks(LHS, % of total)
Global trade and financial linkages
Note: Trade openness is the average of all countries; the share of foreign banks and financial openness are the median OECD country.
• International trade and financial openness.
• Banking sector information.
• Economy-wide regulations.
• Pre-crisis imbalances.
• Survey information.
• Fiscal consolidation.
• The euro area crisis.
14
What factors could be correlated with
recent growth forecast errors?
Ex
pla
inin
g f
ore
cast
err
ors
In the downturn (May 2008 for 2008-09) Relationship
Trade openness -
Foreign bank penetration -
Factors correlated with forecast errors (+/- indicate growth stronger/weaker than expected)
Ex
pla
inin
g t
he
erro
rs
In the recovery (May 2010 for 2010-11 and May 2011 for 2011-12)
Relationship
Better capitalised banks pre-crisis +
Increases in non-performing loans -
Equity price growth +
Improved consumer sentiment +
15
• Forecast error = Outturn - Forecast
• This relationship is similar for average errors over the full period
The downturn was stronger than projected in
more open economies (negative spillovers)
Cumulative growth forecast errors for 2008-09, made in May 2008
Ex
pla
inin
g t
he
erro
rs
16
-16
-12
-8
-4
0
0 50 100
Foreign banks' assets (% total)F
ore
cast
err
or
(%pt)
-16
-12
-8
-4
0
0 100 200Trade openness
Fore
ca
st
err
or
(%p
t)
There were downside surprises in 2010-11 in
countries with lower pre-crisis bank capital
Ex
pla
inin
g t
he
erro
rs
17 Bank capital is the capital adequacy of deposit-takers, measured as
a ratio of total regulatory capital to risk-weighted assets.
-8
-4
0
4
8
8 12 16 20
Bank regulatory capital in 2007 (%)
Fo
reca
st
err
or
(%p
t)
Growth forecast errors for 2010-11, from May 2010
And also where the financial system
was weakening
Ex
pla
inin
g t
he
erro
rs
18
-12
-8
-4
0
4
-5 0 5 10 15
Change in non-performing loans, 2011-12 (%pt)
Fo
reca
st
err
or
(%p
t)
Growth forecast errors for 2011-12, from May 2011
Growth forecast errors over 2007-12 were
larger in more regulated economies
Ex
pla
inin
g t
he
erro
rs
19 Indicators are the OECD product market regulation index and the OECD
measure of the strictness of employment protection (for regular workers)
0
1
2
3
Least regulated Middle Most regulated
Group mean by product market
Group mean by labour market
Degree of regulation for market indicated (2008)
RMSEs of November projections for next year
• IMF: fiscal multipliers under-estimated in
recovery:
– growth weaker than expected in countries with
stronger projected fiscal consolidation.
• Alternatively, actual consolidation could have
been stronger than projected consolidation.
• Growth disappointments also coincided with
the euro area crisis.
• What does the OECD evidence say?
20
Forecast errors, fiscal consolidation and
fiscal multipliers
Ex
pla
inin
g f
ore
cast
err
ors
Ex
pla
inin
g t
he
erro
rs
Growth weaker than projected in countries with
more fiscal consolidation, but only in Europe.
21
Ex
pla
inin
g t
he
erro
rs
Incorrect assumptions about euro crisis and
govt. bond spreads also a source of error at the
same time.
22
• Yes, growth disappointments in countries with
stronger projected consolidation.
• Yes, growth disappointments in countries with
stronger consolidation than projected.
• But only in Europe, and only if Greece is
included.
• The bond spread errors are a more important
source of growth forecast errors
– confirmed by econometric evidence.
23
Forecast errors, fiscal consolidation and
fiscal multipliers: OECD evidence
Ex
pla
inin
g f
ore
cast
err
ors
PUTTING THE FORECAST ERRORS IN CONTEXT
24
An historical perspective: OECD growth forecast
errors in 2009 and 2010 are similar to the early 1970s
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t
25
Recent errors were larger than in the “Great
Moderation” but smaller than the 1970s…
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t
26
0
1
2
3
4
1971-1981 1982-1990 1991-2006 2007-2012
May projection for current year
November projection for next year
May projection for next year
%pt RMSEs of growth forecasts - G7 countries
Unweighted average of errors in G7 economies
…and are not that different if growth volatility
is accounted for
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t
27
0.0
0.5
1.0
1.5
2.0
2.5
1971-1981 1982-1990 1991-2006 2007-2012
May projection for current year
November projection for next year
May projection for next year
RMSEs adjusted for outcomes - G7 countries
Ratio of GDP growth RMSE to standard deviation of GDP growth
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t The forecasts were considerably worse
than usual for some countries
Dark blue and orange indicate statistical significance (at the 10% level) 28
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
CAN FRA DEU ITA JPN GBR USA
Additional effect in 2007-12
Average error 1982-2006
%ptGrowth forecast errors in 2007-12 compared to the past
From May projections for the next year
OECD countries: 2007-12 Accelerations Decelerations
Number in period 56 118
% correct: projections for same year 86 88
% correct: projections for next year 91 50
29
Directional accuracy is good for growth
accelerations, but less so for growth slowdowns
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t
Directional accuracy of May growth projections
G7 countries: 1982-2006 Accelerations Decelerations
Number in period 82 78
% correct: projections for same year 79 83
% correct: projections for next year 74 45
Tools to help identify growth slowdowns are
needed.
However, forecast errors were strikingly
similar across forecasters (also pre-crisis)
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t
30
-6
-4
-2
0
2
4
2007 2008 2009 2010 2011 2012
From the same year
OECD IMF
Consensus EC
%pt
Average errors in forecasts from May for the year
ahead - G7 countries
-6
-4
-2
0
2
4
2007 2008 2009 2010 2011 2012
From one year ahead%pt
Unweighted average for the G7 economies
• They are conditional on a set of assumptions:
– financial market variables, fiscal policy changes,
commodity prices.
• They can point to the need for policy changes
to tackle unsustainable developments:
– If implemented, projection errors may occur.
• Politically-sensitive issues can cause errors:
– A two year projection had to assume the euro area crisis
would diminish.
31
The OECD forecasts are projections rather
than pure forecasts
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t
CHANGES IN FORECASTING PRACTICES
AND PROCEDURES
32
• Greater centralisation of the forecast process.
• Enhanced monitoring of near-term
developments.
• Increased attention on financial sector.
• Enhanced focus on risk assessments and
global spillovers.
33
Post-crisis changes in forecast practices and
procedures at the OECD and elsewhere
On
go
ing
ch
an
ges
• Reflects errors common to all forecasts and
the importance of global financial and trade
interconnections.
– Early view on global developments and risks
and their implications.
– Early guidance via “top down” projections
bringing together information from different
sources and models.
34
Increased centralisation of the forecast
process
On
go
ing
ch
an
ges
• Surveys and high frequency data can provide
early signals of big changes.
– Nowcasting (OECD indicator models).
– Composite leading indicators.
– Evidence from business contacts (forecast
discussions with BIAC and TUAC).
– Use of internet-based indicators (“big data”)
35
Enhanced monitoring of near-term
developments.
On
go
ing
ch
an
ges
– OECD aggregate financial conditions indices
(used forecasts and scenarios).
– Incorporation of broader range of financial
variables in projections.
– Enhanced discussions with internal/external
financial market specialists.
– Incorporating banking sector and global
interconnectedness in macro models.
36
Increased attention on financial sector O
ng
oin
g c
ha
ng
es
• Reflects uncertainty about basic assumptions
behind projections.
– More information about the risks around the main
projections.
– Greater use of quantitative scenario analysis to
illustrate possible outcomes.
– Horizon scanning to plan ahead for unlikely but
potentially costly events.
37
Enhanced focus on risk assessments and
global spillovers
On
go
ing
ch
an
ges
• Forecasting in recent years proved very challenging and
growth has been repeatedly over-estimated.
• Global interconnectedness, structural policy settings and
the health of the banking sector are all related to forecast
errors.
• Errors in assumptions about the speed at which the euro
crisis would ease have been an important source of growth
forecast errors.
• Important changes are now taking place to forecasting
practices and procedures.
38
Summary
SPARE SLIDES
Pro
per
ties
of
the
fore
cast
err
ors
Inflation was under-estimated on
average
Inflation = percentage change in the private consumption deflator 40
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t The accuracy of G7 forecasts over the
long-run
41
Pu
ttin
g t
he
erro
rs i
n c
on
tex
t
The indicator models were helpful
-10
-8
-6
-4
-2
0
2
4
2007:Q1 2008:Q1 2009:Q1 2010:Q1 2011:Q1 2012:Q1
Estimate 1 quarter ahead
Current quarter estimate
Outturn
Quarterly indicator model estimates and GDP growthG7, weighted average%
42