1. Marco Del Negro, Frank Schorfheide, Frank Smets, and Raf Wouters (DSSW) On the Fit of New-Keynesian Models Discussion by: Lawrence Christiano

2. Objective: Provide a scalar measure of the fit of a Dynamic, Stochastic, General Equilibrium Model (DSGE). Apply measure of fit to an empirically important example.

3. Consider a vector autoregression (VAR): Least squares estimation:

4. DSGE model implication for VAR –DSGE model parameters – θ Hybrid model

5. Models: Marginal likelihood: Best fit: Finding: Conclusion: ‘Evidence of model misspecification’

6. Questions Although the marginal likelihood is a sensible way to assess fit in principle… –Compromises are required for tractability –How compelling are the assumptions about likelihood function, priors… How severe is the evidence against the model when –Even if DSGE model were true, unrestricted VAR might fit better in a small sample Is the Hybrid model useful?

7. DSSW Assume the Likelihood of the Data is Gaussian Fit a four-lag, 7 variable VAR using US data, 1955Q4-2006Q1. Compute skewness and kurtosis statistics for each of 7 VAR disturbances

8. There is strong evidence against normality assumption

9. Prior on VAR Parameters Gaussian Likelihood is a function only of VAR parameters: How do DSGE model parameters enter? –They control the priors on VAR parameters:

10. Prior on VAR Parameters… Density In case DSGE model Is true Density in case DSGE model is false

11. Prior on VAR Parameters… Do DSSW priors fairly capture notion that DSGE model might be false? Another possibility: –If preferred DSGE model is false, some other DSGE model is true. –Must specify a prior over alternative DSGE models. Induced priors over VAR parameters likely to be different from Normal/Wishart assumption of DSSW –Problem: Most likely, could not even describe alternative DSGE models, much less assign priors to them! Presumably, this would lead us even further away from DSSW. These concerns about the DSSW priors would be mere quibbles if their approach were the only one to assessing model fit. –But, there are other approaches –More on this later…

12. And DSGE Model Fit Priors for DSGE: Marginal likelihood:

13. Questions How severe is the evidence against the model when To answer this, studied multiple artificial data samples generated from a simple DSGE model

14. Simple (Long-Plosser) Model Setup: Experiment:

15. Results Doing DSSW calculations on artificial data Implications –DSSW evidence of misspecification occurs 1/3 of the time, even though DSGE model is true. –Misspecification of likelihood seems not to matter.

16. Interpretation of Results Why do DSSW find evidence against DSGE model, even when the model is true? One answer: In finite samples, unrestricted VAR often fits substantially better than true VAR implied by DSGE.

17. Interpretation of Results… Interior typically occur in samples where VAR fits substantially better than true model

18. Conclusion DSSW rule: –‘We have evidence of misspecification whenever the peak of the marginal likelihood function is attained at a finite value.’ –with high probability, this rule leads to overly pessimistic assesment of models. What can we learn from about fit of DSGE models? –Requires doing simulation experiments in more elaborate models. –Poses significant computational challenges.

19. Conclusion…. Marginal likelihood provides a sensible measure of fit in principle, however –Assumptions required for tractability render marginal likelihood hard to interpret. –The hybrid model is selected by marginal likelihood criterion – why should it be taken seriously? A less sophisticated, but more transparent and easy to interpret measure of fit: –Out of Sample Root Mean Square Errors.

26. Most likely Model, Other model Prior on model 2: P(M 2 ) Prior on model 1: P(M 1 )

27. Prior on VAR Parameters… The alternative priors would presumably be very different (e.g., multimodal). In practice, we don’t know what other model might be true (this is a basic fact about research!) –How would we even think of priors in this case? –Robust control? Placing priors on VAR parameters conditional on model being false seems very difficult. –Is the DSSW approach the right one? If DSSW approach were the only way to assess model fit, concerns about plausibility of prior would have less force –But, there are other approaches –More on this later…