Maurizio Bovi

Are the Representative Agent’s Beliefs Based on Efficient Econometric Models?

Brussels15 November 2012

EUROPEAN COMMISSION DIRECTORATE GENERAL ECONOMIC AND FINANCIAL AFFAIRS

EU WORKSHOP ON RECENT DEVELOPMENTSIN BUSINESS AND CONSUMER SURVEYS

2

Plan Motivation: several assumptions but little evidence on how

laypeople form expectations

Occasion: Real-Time Data thanks to BoE, Survey Expectations thanks to European Commission

Data Analysis:

Survey: Heterogeneous expectations as Signal/Noise Ratios (SNR) Real Time “Hard” Data: Econometric models and MSE horse race

Contributions:

i) Representative agent’s expectations may be not grounded in optimal econometric models

ii) VAR Analysis of the Expectations Feedback System

(beliefsrealizations) resulting in: “SNR => MSE”

Concluding Remarks and a Tentative Agenda

3

Are the Representative Agent’s beliefs based on efficient statistical models?

People form rational expectations (Muth, 1961):

agents know and use the same “true” model.

People learn the correct model (Evans and Honkapohja, 2001):

agents act as econometricians and relentlessly estimate models.

People are “infected” by professional forecasters’ models (Carroll, 2003):

economists produce forecasts => mass media report forecasts => (more or

less frequently) people read forecasts.

People use the model with the highest fitness (Brock and Hommes, 1997):

People examine different forecasting models switching from one model to

another after a cost-benefit analysis based on relative mean squared

errors. Model uncertainty (i.e. the kind of the optimal model changes with

high frequency), preferences and inertia in the dynamic switching can

create heterogeneous expectations (even to a greater extent wrt sticky

information, Branch, 2004 & 2007).

4

Issues People form rational expectations (Muth, 1961):

Several authors find evidence rejecting this assumption.

People act as smart econometricians (Evans and Honkapohja, 2011):

This is the typical assumption maintained by the adaptive learning

literature. Finding evidence on that is one of my goals.

People are “infected” by professional forecasters (Carroll, 2003):

If so, why household surveys are still among the most watched market

movers even among professional forecasters?

Predictor Choice Approach (Brock and Hommes, 1997):

To be able to calculate the relative success of their own choice, agents

must know the success of all competing models: individuals have already

paid the computational cost. Then, the question can be asked whether the

determinant of the choice should deal with forecasting accuracy only.

Also, “preferences” towards a single model are at odds wrt both the

dynamic switching and learning activities.

5

Alternative Views on the Expectations Feedback System

Pigou (1927), Keynes (1936), Simon (1957), Tversky and

Kahneman, (1974)

Psychology matters and not-econometrically-based factors is what

surveys could/should capture (Katona, 1944, 1975; Bovi, 2009)

Ludvigson (2004)

Many empirical papers have been looking, with some success, for

the additionaladditional information content of survey expectations, whereby

the adjective “additional” stands exactly for extra economic

elements and/or independent information

Cass and Shell (1983)

Heterogeneous beliefs – e.g. in models with self-fulfilling prophecies

sunspot equilibria - can drive macroeconomic outcomes.

6

Key Questions and Logic

Are representative agents’ (=>laypeople) beliefs based on optimal econometric models? Do heterogeneous beliefs Granger-cause the predictive power of efficient econometric models, or vice versa? Here the logic:

Think about an economy whereas a simple model turns out to be the best predictor for many years, but survey-declared expectations do not converge.

Think about an economy whereas evidence points out that

SNR “precede” MSE, but not vice versa.

All in all, the general validity of assuming best model-based expectations would be weakened.

7

2. Survey Data

Source: Business Surveys Unit of the European Commission

Sample Period: January 1985 – March 2009 (simple mean for quarterly data)

Features: No genuine panel (bad), continuously refined (good), Laypeople (good: it is likely that economists use econometric models to forecast)

Query: “How do you expect the general economic situation in the country to develop over the next 12 months?”

Reply Options: It will…

…get a lot better (=LB); …get a little better (=B); …stay the same (=E); …get a little worse (=W); …get a lot worse (=LW); don't know (=N).

LB, B, E, etc., are the shares of respondents having chosen the corresponding option so that they sum up to one.

8

2.1 Survey Expectations Signal/Noise Ratios

9

2.1a Survey Expectations Signal/Noise Ratios

10

2.1b Survey Expectations Signal/Noise Ratios

Unlike the previous methods, the IQV does not account for the ordered nature of the data.

11

2.2 Survey Expectations and Reality

-6.0

-4.0

-2.0

0.0

2.0

4.0

6.0

8.0

0.72

0.76

0.80

0.84

0.88

0.92

0.96

1.00

86 88 90 92 94 96 98 00 02 04 06 08

UK GDP annual grow th rateSignalNoise (rhs, 1=fully heterog. expect.)

12

3. Real Time Data

When comparing econometric models and survey expectations, the former should be estimated in real time (and known by the representative agent: VAR before 1980?)

Why? Because otherwise one is assuming that representative agents use information which will be available only in future dates or that they have remarkably good foresight about data revisions

Trivial? Not so much: attention to actually available data is becoming widespread in the literature only recently (Croushore, 2010). As for the techniques, sometimes laypeople are asked to act as leading econometricians.

13

3.1 BoE Real Time Data Set

Real GDP: the first disposable vintage (released in 1976Q1)

covers the period 1955Q1-1975Q4,the second was published a quarter later and covers the

period 1955Q1-1976Q1... the last release I use here covers the period 1955Q1-

2009Q1 Prices (GDP and Private Consumption Deflators): the first release includes data from 1970Q1 to 1989Q4

(released in Jan. 1990)… the last 1970Q1 to 2009Q1 Interest rate (3M Treasury Bill Rate):no data revision, available since 1957Q1

14

3.2 The Competing Models

All models estimated recursively, AR and VAR models via rolling regressions, too

15

3.3 Real-Time Estimation

I sequentially estimate the mentioned models using real time data:

Example:

in 1976Q1 (=t+1) the very first yt time series, running from 1956Q1

to 1975Q4 (=t), is made available. The 1st four-steps-ahead prediction refers

to 1976Q4 (=t+4). To compare this with its realization we have to wait

until 1977Q1 (=t+5), when the actual data for 1976Q4 is eventually released.

According to survey and hard data availability, • Univariate models generate 97 recursive forecasts, from 85Q1 to 09Q1• VAR models produce 74 recursive forecasts, from 90Q4 to 09Q1

16

3.3a Real-Time Estimation

AE model. Each quarter grid search over all (0,1], with step size 0.04, choosing the value of gamma that minimizes the squared forecasting errors.

Rolling windows estimation:• Minimum window size = 32 quarters;• Max window size = 56 quarters.

So, to find the optimal (MSE-minimizing) window size, I perform 24 separate rolling regressions for each (non naïve) model.

17

3.4 Real-Time Forecasting Metric

To mimic the forecasting exercise elicited in the surveys, I forecast “the next twelve months”, computing the quarterly squared errors as follows:

MSEt+5= (t+5yt+4 – t+1yet+4)2

where:

yt = (GDPt-GDPt-4)/GDPt-4

t+1yet+4 = Expected value of y in t+4 based on the vintage released in t+1

t+5yt+4 = Actual value of y in t+4 as reported by the vintage released in t+5

Due to data availability, the first useful squared error is

MSE85:Q1 for univariate models

MSE90:Q1 for multivariate models

18

3.5 Real-Time Econometric Models Forecasting Ability

Model 85Q1-89Q4 90Q1-96Q2 96Q3-02Q4 03Q1-09Q1 90Q1-09Q1

RW 3.73 5.34 1.73 4.31 3.8

AE 1.17 2.49 0.42 3.09 2.01

AR1 1.98 5.11 0.98 3.72 3.27

AR1 rolling 1.8 3.52 0.9 3.71 2.73

VARPC1 (y,) NA 6.64 1.45 4.24 4.11

VARPC1 rolling NA 4.29 1.27 3.79 3.12

VARPC2 (y,) NA 7.07 1.66 4.56 4.43

VARPC2 rolling NA 3.71 1.34 3.74 2.94

VAR1 (y,, r) NA 6.97 3.16 5.82 5.32

VAR1 rolling NA 4.1 1.01 3.12 2.75

VAR2 (y,, r) NA 6.52 3.18 5.98 5.24

VAR2 rolling NA 3.45 0.95 3.48 2.64

Sample averages of the MSE of the corresponding models

19

Naive Expectations Forecasting Accuracy (MSE) as a Proxy of the Great Moderation

0

2

4

6

8

10

12

86 88 90 92 94 96 98 00 02 04 06 08

MSE_RWMSE_AE

20

3.6 Relative Forecasting Ability wrt RW

t-stat of the constant term in the “(MSE_J-MSE_RW)=const” regression. When a curve is below the -1.69 horizontal line (indicating the 5% p-value), then the corresponding model “J” dominates the benchmark.

-4

-3

-2

-1

94 96 98 00 02 04 06 08

AE

-2

-1

0

1

2

94 96 98 00 02 04 06 08

AR1

-2

-1

0

1

2

94 96 98 00 02 04 06 08

AR1 rolling

-2

-1

0

1

2

94 96 98 00 02 04 06 08

VARPC1

-4

-3

-2

-1

0

1

94 96 98 00 02 04 06 08

VARPC1 rolling

-2

-1

0

1

2

94 96 98 00 02 04 06 08

VARPC2

-4

-3

-2

-1

0

1

94 96 98 00 02 04 06 08

VARPC2 rolling

-2-101234

94 96 98 00 02 04 06 08

VAR1

-4

-3

-2

-1

0

1

94 96 98 00 02 04 06 08

VAR1 rolling

-2-101234

94 96 98 00 02 04 06 08

VAR2

-4

-3

-2

-1

0

1

94 96 98 00 02 04 06 08

VAR2 rolling

21

3.6a Relative Forecasting Ability wrt AE

-2

0

2

4

6

94 96 98 00 02 04 06 08

AR1

-2-101234

94 96 98 00 02 04 06 08

AR1 rolling

-2-101234

94 96 98 00 02 04 06 08

VARPC1

-2

0

2

4

6

94 96 98 00 02 04 06 08

VARPC1 rolling

-2-101234

94 96 98 00 02 04 06 08

VARPC2

-2-101234

94 96 98 00 02 04 06 08

VARPC2 rolling

-2-101234

94 96 98 00 02 04 06 08

VAR1

-2

-1

0

1

2

3

94 96 98 00 02 04 06 08

VAR1 rolling

-2-101234

94 96 98 00 02 04 06 08

VAR2

-2

-1

0

1

2

3

94 96 98 00 02 04 06 08

VAR2 rolling

t-stat of the constant term in the “(MSE_J-MSE_AE)=const” regression. When a curve is below the -1.69 horizontal line (indicating the 5% p-value), then the corresponding model J dominates the benchmark.

22

01

23

45

67

89

1011

120

12

34

56

78

910

1112

01

23

45

67

89

1011

120

12

34

56

78

910

1112

01

23

45

67

89

1011

12-0,5

-0,4

-0,3

-0,2

-0,1

0

0,1

IQV CP3 CP BAL3 BAL

| | | | | | | | | | | | | | | | | | | | | | | | | | | | |

| | | | | | | | | | | | | | | | | | | | | | | | | | | | |

| | | | | | | | | | | | | | | | | | | | | | | | | | | | |

| | | | | | | | | | | | | | | | | | | | | | | | | | | | |

i =

______________________________________________________________________________________________________________________________

01

23

45

67

8910

1112

01

23

45

67

8910

1112

01

23

45

67

8910

1112

01

23

45

67

89

1011

120

12

34

56

78

910

1112

-0,4

-0,3

-0,2

-0,1

0

0,1

0,2 | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

IQV

| | | | | | | | | | | | | | | | | | | | | | | | | | | | |

| | | | | | | | | | | | | | | | | | | | | | | | | | | | |

| | | | | | | | | | | | | | | | | | | | | | | | | | | | |

CP3 CP BAL3BAL

i =

________________________________________________________________________________________________________________________________

Upper Panel: correlation(SNRt-i;MSE_AEt); Lower Panel: correlation(SNRt;MSE_AEt-i)

i=0,…,12. NB: Correlations within (-0.2;+0,2) are statistically zero.

23

4. Where Do We Stand?

So far, we have:

• Twelve time series made up by quarterly MSE (85/90Q1-09Q1) pointing out that the AE model outperforms all the others all the times. Despite of that:

• Five quarterly signal-to-noise ratios (85Q1-09Q1) point out that great variety in expectations exists and persists. Then:

• SNR and MSE seems to co-move according to SNR => MSE. This calls for more formal tests.

Specifically, I estimate a battery of bivariate VAR made up by one SNR and one MSE (stemming from the best model) in order to perform Granger, FE variance decomposition and Geweke tests

24

4.1 Granger Causality

25

4.2 No Instantaneous Feedback

26

5 Comments on Results

Granger-causality, Geweke’s instantaneous feedback and VD tests:

• Past, present and future values of MSE are not a determinant of the dispersion across agents' expectations. Instead,

• Divergent expectations significantly affect the forecasting accuracy of optimal econometric models. Then, interpreting the MSE as a proxy of volatility,

• Disagreement across laypeople’s expectations Granger-causes macroeconomic uncertainty, but not vice versa.

These outcomes are in line with the literature on the extra information content of the surveys, and contrast with some of the assumptions behind i) the adaptive learning, ii) the predictor choice, and iii) the epidemiological frameworks.

27

5a. Comments on Results

Negative correlations A more perturbed signal coming from the surveys leads to: i) higher model-based MSE; ii) greater macroeconomic uncertainty

No model uncertainty, but high and persistent disagreementEvidence strongly indicates that a relatively simple AE predictor outperforms all the other proposed models all the time.Evidence strongly indicates that laypeople expectations are persistently heterogeneous.

Results are robust to several SNR and models, including univariate and multivariate models even estimated via optimal-size rolling windows.

28

6 Concluding Remarks

Should heterogeneity depend on model uncertainty only, UK citizens’ expectations should likely converge

Should lay consumers’ expectations be best-model-based, then SNRi) should not significantly help predict optimal model-based MSE and ii) should not follow univariate processes wrt past information about MSE

Since data show opposite findings, then there must be some additional explanation behind the formation of disparate expectations

The identification of these disagreement-widening factors is in my research agenda

29

THANK YOU!THANK YOU!