1. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Systematic Errors Thomas Jung European Centre for Medium-Range Weather Forecasts

2. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Scope of the Lecture Question: Do we still have significant systematic errors in the ECMWF forecasting system? If so, what are the main problems? How do systematic errors grow? How did systematic errors evolve over the years? How well do we simulate specific phenomena (e.g., blocking, extratropical cyclones)? Does increasing resolution help? Which techniques can be used to understand systematic error?

3. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Introduction Two principal sources of forecast error: Uncertainties in the initial conditions (observational error) Model error Two kinds of forecast error Random error (model+initial error) Systematic error (model error*)

4. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Concept of Systematic Error Relatively straightforward to compute BUT can be difficult to identify the origin of errors Moreover there are pitfalls: finite length (significance tests help partially) apparent systematic error for short time series (loss of predictability) observations might be biased

5. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Loss of Predictability and Systematic Error Observed Anomaly Systematic Error when predictability is lost Fingerprint of loss of predictability: Systematic error resembles the opposite of the observed anomaly Observed Forecast

6. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Example: Z500 DJF 2005/06 Observed AnomalyMean Error @ D+10 Spatial correlation=-0.78

7. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Uncertainty of our Truth

8. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Data Medium-range forecasts Primarily hindcasts from ERA-40 and ERA-Interim (robust statistics). Seasonal integrations: 13 month long integrations started on 1 st November of each of the years 1962-2005 (or a subset of this period). Diagnosis of standard seasons DJF, MAM, JJA, SON (errors had at least 1 month to grow, asymptotic errors). Most experiments at T L 159 with 91 levels in the vertical. Observed lower boundary conditions (uncoupled integrations). Observational data: ERA-40 Other observational data sets.

9. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Systematic Error Growth How do systematic errors grow throughout the forecast?

10. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Systematic Z500 Error Growth from D+1 to D+10

11. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Systematic Z500 Errors: Medium-Range and Beyond D+10 ERA-Interim Asymptotic: 31R2

12. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Evolution of Systematic Error How did systematic errors evolve throughout the years?

13. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Evolution of D+3 Systematic Z500 Errors 1983-19871993-1997 2003-2007

14. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Evolution of Systematic Z500 Errors: Model Climate 35R133R132R332R2 32R131R130R129R2

15. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Systematic Z500 Errors: Impact of Recent Changes ControlOld ConvectionOld TOFD Old Vertical DiffOld RadiationOld Soil Hydrology

16. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Phenomena

17. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Blocking Methodology L H L

18. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Blocking Frequency Errors (23r4) DJF 1990-2001 ERA-40 D+1 D+4 D+7 D+10

19. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Blocking Frequency Errors (31R2) DJF 1990-2001 ERA-40 D+1 D+4 D+7 D+10

20. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Blocking Frequencies: DJF 1990-2005 ERA40 33R1 32R3 29R2-32R2

21. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. New Blocking Diagnostic Feature tracking software from Kevin Hodges Track positive stream function anomalies at 1000 hPa Some filtering: T3-T12 Diagnostics: Number, frequency, lifetime, …

22. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Frequency of Anti-cyclones (1962-2005 DJF) ERA40 33R1-ERA40 31R1-ERA40 33R1-31R1 Long-lived anticyclones only (> 2 days)

23. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Average Lifetime of Anticyclones (1962-2005 DJF) ERA40 33R1-ERA40 31R1-ERA40 33R1-31R1

24. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Extratropical Synoptic Systems Two appoaches: Compute standard deviation of highpass or bandpass filtered time series (Eulerian approach). Track individual systems (Lagrangian approach).

25. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Observed Tracks of Long-lived Cyclones >2 days

26. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Sensitivity to Horizontal Resolution: 29R1

27. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Sensitivity to Horizontal Resolution: 31R1

28. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Number of Extratropical Cyclones (DJFM) Lifetime > 1day

29. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Computational Effort 12% of all CPUs on HPCE cluster Wall clock time about 20 hours About 70 times more expensive than T L 95! 1 Integration (151 days) @ T L 511L91:

30. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Synoptic Activity Error: Evolution 35R1 32R3 31R1 32R2 30R1 33R1

31. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. The Madden-and Julian Oscillation

32. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Schematic of the MJO From Madden and Julian (1994)

33. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Near Global Impact of the MJO (Precipitation) Conv.: Indian OceanConv.: Maritime Continent Conv.: Central PacificConv.: WH/Africa

34. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. The Madden and Julian Oscillation ERA-40 ECMWF Model

35. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Conclusions (1) Main systematic errors: Concept of systematic error is very straightforward. But there are pitfalls: Short time series (sampling issues + loss predictability). Uncertainty about the true state of the atmosphere. Systematic error are a clear sign of model error. Understanding systematic error is challenging.

36. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Conclusions (2) Main systematic errors: Do we still have systematic errors in the ECWMF model? Yes, we do. Most systematic errors have been substantially reduced in recent years. Some key-challenges remain, however!

37. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Conclusions (3) Main systematic errors: Tropical circulation + hydrological cycle Madden-Julian Oscillation Indian Summer Monsoon Quasi-Biennial Oscillation Euro-Atlantic blocking Synoptic activity (severe wind storms) Others (surface related fields) Key-challenges:

38. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Further Reading Bechtold, P., M Koehler, T. Jung, M. Leutbecher, M. Rodwell and F. Vitart, 2008: Advances in simulating atmospheric variability with the ECMWF model: From synoptic to decadal time scales. Quart. J. Roy. Meteor. Soc., 134, 1337-1351. Jung, T. and A.M. Tompkins, 2003: Systematic Errors in the ECMWF Forecasting System. ECMWF Technicial Memorandum 422. http://www.ecmwf.int/publications/library/do/references/list/14 Jung, T., A.M. Tompkins, and R.J. Rodwell, 2004: Systematic Errors in the ECMWF Forecasting System. ECMWF Newsletter, 100, 14-24. Jung, A.M. Tompkins, and R.J. Rodwell, 2005: Some Aspects of Systematic Errors in the ECMWF Model. Atmos. Sci. Lett., 6, 133-139. Jung, T. and co-authors, 2009: The ECMWF model climate: Recent progress through improved physical parametrizations. ECMWF Seminar Proceedings on Parameterization of Subgrid-scale Processes, 233-249. Available from the ECMWF website. Jung, T., 2005: Systematic Errors of the Atmospheric Circulation in the ECMWF Model. Quart. J. Roy. Meteor. Soc., 131, 1045-1073. Jung, T., S.K. Gulev, I. Rudeva and V. Soloviov, 2006: Sensitivity of extratropical cyclone characteristics to horizontal resolution in the ECWMF model. Quart J. Roy. Meteor. Soc., 132, 1839-1857. Hoskins, B.J. and K.I. Hodges, 2002: New Perspectives on the Northern Winter Storm Tracks. J. Atmos. Sci., 59, 1041-1061. Koehler, M., 2005: ECMWF Technicial Memorandum 422. http://www.ecmwf.int/publications/library/do/references/list/14 Palmer, T.N., G. Shutts, R. Hagedorn, F. Doblas-Reyes, T. Jung, and M. Leutbecher, 2005: Representing Model Uncertainty in Weather and Climate Prediction. Ann. Rev. Earth. Planet Sci., 33, 163-193. Rodwell, MJ and T Jung, 2008: Understanding the local and global impacts of model physics changes: An aerosol example. Quart. J. Roy. Meteor. Soc., 134, 1479-1497.

39. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Recent Model Changes 29R228 Jun 2005Modifications to convection 30R11 Feb 2006Increased resolution (L60 to L91) 31R112 Sep 2006 Revised cloud scheme (ice supersaturation + numerics); implicit computation of convective transports; introduction of orographic form drag scheme; revised GWD scheme 32R1 not operational New short-wave radiation scheme; introduction of McICA cloud- radiation interaction; MODIS aerosol; revised GWD scheme; retuned ice particle size 32R25 Jun 2007Minor changes to forecast model 32R36 Nov 2007 New formulation of convective entrainment and relaxation time scale; reduced vertical diffusion in the free atmosphere; modification to GWD scheme (top of the model); new soil hydrology scheme 33R13 Jun 2008Slightly increased vertical diffusion; increased orographic form drag; retuned entrainment in convection scheme; bugfix scaling of freezing term in convection scheme; changes to the surface model

40. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Sensitivity to Model Formulation

41. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Systematic Errors: AMIP Models

42. Training Course Presentation, 16 th March 2009 – Thomas Jung: Systematic Model Errors. Demeter Models (DJF Precipitation) CERFACS LODYC MPIMetOffice Meteo- France ECMWF