Marco Del Negro, Frank Schorfheide, Frank Smets, and Raf Wouters (DSSW)

On the Fit of New-Keynesian Models

Discussion by:

Lawrence Christiano

Objective:

• Provide a scalar measure of the fit of a Dynamic, Stochastic, General Equilibrium Model (DSGE).

• Apply measure of fit to an empirically important example.

• Consider a vector autoregression (VAR):

• Least squares estimation:

1m yt

1 k xt

km

1m ut , Eutut

XX 1XY

1TY X

Y X

• DSGE model implication for VAR– DSGE model parameters – θ

• Hybrid model

XX 1TEXX, XY 1

TEXY, YY 1

TEYY

XX 1 XY , YY YX XX 1 XY

, XX T XX 1XY T XY

, ~analogously defined.

• Models:

• Marginal likelihood:

• Best fit:

• Finding:

• Conclusion: ‘Evidence of model misspecification’

DSGE model

small VAR model

small Hybrid Model

L Y, parameters

LY|parameters, Pparametersdparameters

maxsmall

L Y,

0.75,1.5

Questions• Although the marginal likelihood is a sensible

way to assess fit in principle…

– Compromises are required for tractability

– How compelling are the assumptions about likelihood function, priors…

• How severe is the evidence against the model when

– Even if DSGE model were true, unrestricted VAR might fit better in a small sample

• Is the Hybrid model useful?

DSSW Assume the Likelihood of the Data is Gaussian

• Fit a four-lag, 7 variable VAR using US data, 1955Q4-2006Q1.

• Compute skewness and kurtosis statistics for each of 7 VAR disturbances

• There is strong evidence against normality assumption

Disturbance kurtosis skewness

statistic, s probs s| normal statistic, s probs s| normal

logCt/Yt 1.40 0.18% -0.17 84.64%

logI t/Yt 1.59 0.12% 0.07 33.04%

PCE inflationt 0.79 2.00% 0.41 0.68%

logYt/lt logWt/Pt 1.04 0.66% 0.39 1.22%

R t 11.01 0.00% 1.62 0.00%

logl t 1.27 0.40% 0.11 26.46%

GDP growtht 1.85 0.02% -0.02 55.00%

Prior on VAR Parameters• Gaussian Likelihood is a function only of

VAR parameters:

• How do DSGE model parameters enter?– They control the priors on VAR parameters:

LY| ,

P , | ,

Prior on VAR Parameters…

DensityIn case DSGE modelIs true

Density in caseDSGE model isfalse

Prior on VAR Parameters…• Do DSSW priors fairly capture notion that DSGE model might be

false?

• Another possibility:

– If preferred DSGE model is false, some other DSGE model is true.

– Must specify a prior over alternative DSGE models. Induced priors over VAR parameters likely to be different from Normal/Wishart assumption of DSSW

– Problem: Most likely, could not even describe alternative DSGE models, much less assign priors to them! Presumably, this would lead us even further away from DSSW.

• These concerns about the DSSW priors would be mere quibbles if their approach were the only one to assessing model fit.

– But, there are other approaches– More on this later…

And DSGE Model Fit• Priors for DSGE:

• Marginal likelihood:

P

L Y,

, LY| , P , | , P d , d

L Y,

Huge integration problem, made trivial by:

Normal assumption on LY| , Normal/Wishart assumptions P , | ,

,

LY| , P , | , d , P d

Questions

• How severe is the evidence against the model when

• To answer this, studied multiple artificial data samples generated from a simple DSGE model

Simple (Long-Plosser) Model• Setup:

• Experiment:

E0 t 0

0.99t logCt expxt 1 l t

1 ,

Ct Kt 1 Kt1/3expzt l t 2/3 Yt,

xt, zt ~iid mean zero, variance x2, z2

, x2, z2 , true 1,0.022, 0. 022

Uniform priors: ~0,2, x2, z2~0.0001,0.0007

200 observations on: log Ytl t

, logKt 1 .

Results• Doing DSSW calculations on artificial data

• Implications

– DSSW evidence of misspecification occurs 1/3 of the time, even though DSGE model is true.

– Misspecification of likelihood seems not to matter.

. 33, 0.5, 0.75, 1, 1.25, 1.5, 2, 5

Prob 5 E | 5

Normal Disturbances 33.4 1.43

Kurtotic Disturbances 34.9 1.33

Note:Monte Carlo standard error on prob, 1.5

Interpretation of Results

• Why do DSSW find evidence against DSGE model, even when the model is true?

• One answer: In finite samples, unrestricted VAR often fits substantially better than true VAR implied by DSGE.

Interpretation of Results…• Interior typically occur in samples

where VAR fits substantially better than true model

E LR test of DSGE model|

0.33 28.8

0.50 17.2

0.75 15.3

1.00 13.23

1.25 12.2

2.00 10.6

5.00 7.5

Conclusion• DSSW rule:

– ‘We have evidence of misspecification whenever the peak of the marginal likelihood function is attained at a finite value.’

– with high probability, this rule leads to overly pessimistic assesment of models.

• What can we learn from about fit of DSGE models?

– Requires doing simulation experiments in more elaborate models.

– Poses significant computational challenges.

Conclusion….• Marginal likelihood provides a sensible measure

of fit in principle, however

– Assumptions required for tractability render marginal likelihood hard to interpret.

– The hybrid model is selected by marginal likelihood criterion – why should it be taken seriously?

• A less sophisticated, but more transparent and easy to interpret measure of fit:

– Out of Sample Root Mean Square Errors.

Most likelyModel,

P |M1 Other model

P|M2

Prior on model 2: P(M2 )

Prior on model 1: P(M1 )

P , P ,

marginal prior over VAR parameters: P ,

Prior on VAR Parameters…• The alternative priors would presumably be very different (e.g.,

multimodal).

• In practice, we don’t know what other model might be true (this is a basic fact about research!)

– How would we even think of priors in this case?– Robust control?

• Placing priors on VAR parameters conditional on model being false seems very difficult.

– Is the DSSW approach the right one?

• If DSSW approach were the only way to assess model fit, concerns about plausibility of prior would have less force

– But, there are other approaches– More on this later…