1 LAM EPS Workshop, Madrid, 3-4 October 2002 Ken Mylne and Kelvyn Robertson Met Office Poor Man's EPS experiments and LAMEPS plans at the Met Office

2 LAM EPS Workshop, Madrid, 3-4 October 2002 Why PEPS (Poor Man’s EPS)? Storms of Dec 1999 over Europe were poorly forecast by most deterministic models, even at 24h –Need for effective short-range ensemble to reduce risk of missing severe weather events Existing operational ensembles (eg ECMWF) designed for medium-range (3-10days) –some evidence of poor performance for severe events in short-range PEPS is an ensemble formed by combining the operational output from several NWP centres –provides a relatively cheap way of obtaining short-range ensemble forecasts

3 LAM EPS Workshop, Madrid, 3-4 October 2002 Why PEPS might work Multi-model multi-analysis ensemble –experiments in USA have shown this is important (eg Hou et al, 2001; Stensrud et al, 1999) Random sampling of initial condition errors –may be important for estimating probabilities at short-range Previous studies (eg Ziehmann, 2000) have shown encouraging results

4 LAM EPS Workshop, Madrid, 3-4 October 2002 Preliminary system 9 models Low-res (5x5°) H500 and pmsl only Output every 24h Data stored and used by VT, not DT

5 LAM EPS Workshop, Madrid, 3-4 October 2002 Verification - Brier Skill Brier Skill Scores, using the ECMWF EPS as ref. Several PEPS configurations –all available models –one model removed (all versions) –all plus 6 members of EPS –reduced combinations Range of PMSL thresholds 126 days from 7 th Feb to 12 th June 2001

6 LAM EPS Workshop, Madrid, 3-4 October 2002 Hi-Res PEPS Success of the preliminary system encouraged us to set up a much larger PEPS system: Larger ensemble –around 15 members from 9 models Higher resolution –tests at 1.25x1.25° –output every 12h More fields –PMSL –H500 –T850 –2m Temp –10m Windspeed –Precipitation

7 LAM EPS Workshop, Madrid, 3-4 October 2002 Data Exchange 9 centres agreed to supply forecast data Data are pulled from FTP sites in near-real time –European data via ECMWF fast link –Other centres via the internet –Met Office UM –ECMWF –DWD –Meteo-France –BoM –JMA –KMA –CMC –NCEP –Russia

8 LAM EPS Workshop, Madrid, 3-4 October 2002 Brier Skill - Winter DJF 2001/02 Results similar to preliminary experiments Reference EPS is 12 hours older due to late data cut-off –provides the gain which could be achieved operationally

9 LAM EPS Workshop, Madrid, 3-4 October 2002 Effect of 12h Advantage Re-ran verification without giving PEPS the 12h advantage Apparent PEPS skill mostly comes from the 12h advantage Without: –No skill at T+24 –Slight advantage at T+84 With 12hWithout 12h

10 LAM EPS Workshop, Madrid, 3-4 October 2002 BSS - Different Weather Parameters PMSLH500T850T 2m10m WS T+24 T+72 Results similar for all weather parameters:-

11 LAM EPS Workshop, Madrid, 3-4 October 2002 BSS - PMSL in Regions N. Hem.EuropeN. Am.S. Hem. PMSL results poor over S. Hemisphere. T+72 T+24

12 LAM EPS Workshop, Madrid, 3-4 October 2002 BSS - 2m Temperature in Regions N. Hem.EuropeN. Am.S. Hem. T2m results poor over S. Hemisphere. Best over continents but still poorer than EPS. T+72 T+24

13 LAM EPS Workshop, Madrid, 3-4 October 2002 BSS - Wind Speed in Regions N. Hem.EuropeN. Am.S. Hem. Benefit for more extreme events in all regions:- T+72 T+24

14 LAM EPS Workshop, Madrid, 3-4 October 2002 Rank Histograms PMSL over Northern Hemisphere –over-spread at 24-48h –good spread but slight bias at longer lead- times –EPS underdispersive at all times to T+120

15 LAM EPS Workshop, Madrid, 3-4 October 2002 Rank Histograms Focus on over- spreading at T –Northern hemisphere average hides strong regional bias over Europe –still some over-spreading –And an opposite regional bias over N. America

16 LAM EPS Workshop, Madrid, 3-4 October 2002 Rank Histograms Focus on over- spreading at T –Southern hemisphere shows stronger over-spreading –probably due to analysis biases Difficult to separate ensemble spread from differences in model biases Some apparent over-spreading may be due to biases in the verifying ECMWF analysis Need for bias correction

17 LAM EPS Workshop, Madrid, 3-4 October 2002 Rank Histograms Weather parameters PMSL 500hPa Height –Strong bias (analysis?) –Some over-spreading T850 –Over-spreading

18 LAM EPS Workshop, Madrid, 3-4 October 2002 Rank Histograms Weather parameters PMSL 2m Temperature –Over-spreading 10m Wind Speed –Over-spreading –Bias

19 LAM EPS Workshop, Madrid, 3-4 October 2002 Reliability Diagrams PMSL<970mb over Northern Hemisphere –reliability good for PEPS and for EPS

20 LAM EPS Workshop, Madrid, 3-4 October 2002 Reliability Diagrams H500<480dm over Northern Hemisphere –some general under- forecasting - possibly bias in ECMWF analysis, as seen in Rank Histograms

21 LAM EPS Workshop, Madrid, 3-4 October 2002 Reliability Diagrams 2m Temperature –<260 deg C –better reliability than EPS for all thresholds –<280 deg C –<300 deg C

22 LAM EPS Workshop, Madrid, 3-4 October 2002 Conclusions on PEPS PEPS advantage over EPS was due to the 12h lag applied to EPS –little scientific advantage of PEPS method at T+24 –slight advantage at T+84 (multi-model?) PEPS over-spread at short-range –regional biases make interpretation difficult –some evidence for better reliability for extreme events Experiments with bias-corrected PEPS should clarify results –set up to run over the coming winter

23 LAM EPS Workshop, Madrid, 3-4 October 2002 Plans for LAMEPS The Met Office is devising plans for a short-range ensemble based on a LAM covering the Atlantic and Europe. Aims: –Risk assessment for rapid cyclogenesis –Uncertainty of sub- synoptic systems –assess probability forecasts of precipitation, low cloud and visibility –LBCs for future storm- scale ensembles

24 LAM EPS Workshop, Madrid, 3-4 October 2002 LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: Initial conditions Model physics parametrizations Lateral boundaries Surface parameters

25 LAM EPS Workshop, Madrid, 3-4 October 2002 LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: Initial conditions Model physics parametrizations Surface parameters Lateral boundaries

26 LAM EPS Workshop, Madrid, 3-4 October 2002 Initial Condition Perturbations Options: Singular vectors (as used at ECMWF) Error breeding (Toth and Kalnay, 1993) (as used at NCEP) Ensemble data assimilation (CMC, Houtekamer et al, 1996) Ensemble Kalman Filter (Bishop et al, 2001) Multi-analysis (INM)

27 LAM EPS Workshop, Madrid, 3-4 October 2002 Initial Condition Perturbations Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Maximise ensemble growth over early forecast range (48h at ECMWF) Possibility of combining SVs optimised at 6h, 12 and 18h (Hollingsworth, personal communication) Some evidence that SVs only provide reliable probabilities for severe weather events well after the optimisation period

28 LAM EPS Workshop, Madrid, 3-4 October 2002 Early Warnings of Severe Weather from EPS Verification of severe weather warnings based on the EPS –Discrimination of events is best at 4 days (ROC) –Better discrimination is independent of calibration –Reliability is best at day 4 and useless at days days 1 day2 days 3 days

29 LAM EPS Workshop, Madrid, 3-4 October 2002 Early Warnings -Brier Skill Scores Brier Skill also tends to increase after day 2. Heavy RainSevere Gales

30 LAM EPS Workshop, Madrid, 3-4 October 2002 Initial Condition Perturbations Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Relatively simple to implement Identifies modes growing rapidly at analysis time –may provide a more random sampling in the early forecast But… bred vectors are not orthogonal –tend to converge –not worth running more than 5-8 cycles

31 LAM EPS Workshop, Madrid, 3-4 October 2002 Initial Condition Perturbations Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Multiple data assimilation cycles with perturbed observations –computationally expensive Accounts for model errors Monte-Carlo method –random sampling, so should provide reliable probabilities In practice did not perform very well at CMC –insufficient spread to scale with forecast errors

32 LAM EPS Workshop, Madrid, 3-4 October 2002 Initial Condition Perturbations Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Various configurations exist Theoretically optimal –not tested in full NWP models –difficulties with some obs types –computationally expensive Ensemble Transform Kalman Filter (Bishop et al, 2001) may provide the best system in the long-term

33 LAM EPS Workshop, Madrid, 3-4 October 2002 Initial Condition Perturbations Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Relatively cheap and simple –reliability may be a problem Accounts for model errors No attempt to identify rapidly growing modes Monte-Carlo method –random sampling, so should provide reliable probabilities PEPS results suggest: –over-spreading –need for bias corrections

34 LAM EPS Workshop, Madrid, 3-4 October 2002 Initial Condition Perturbations Singular vectors Error breeding Ensemble data assimilation Ensemble Kalman Filter Multi-analysis Initially we will use Error Breeding Later we hope to develop EnKF

35 LAM EPS Workshop, Madrid, 3-4 October 2002 LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: Initial conditions Model physics parametrizations Surface parameters Lateral boundaries

36 LAM EPS Workshop, Madrid, 3-4 October 2002 Model Physics Perturbations Again many options… main priorities: Convection Cloud/microphysics –impact on radiation Surface roughness

37 LAM EPS Workshop, Madrid, 3-4 October 2002 Model Physics Perturbations Approaches: Multi-model –effective –opportunity for effective collaboration Multi-scheme –eg. Kain-Fritsch or Betts-Miller convection Perturbed tendency –as used at ECMWF Stochastic physics schemes –conceptually and theoretically elegant –research required - role for universities

38 LAM EPS Workshop, Madrid, 3-4 October 2002 LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: Initial conditions Model physics parametrizations Surface parameters Lateral boundaries

39 LAM EPS Workshop, Madrid, 3-4 October 2002 Surface Parameters Surface Roughness –fixed but uncertain - perturb between members –variable over sea –impact through windspeed, heat and moisture fluxes Soil moisture, SST, snow cover etc –analysed –could be perturbed randomly

40 LAM EPS Workshop, Madrid, 3-4 October 2002 LAMEPS Perturbation Strategy To be fully effective LAMEPS will need perturbations to: Initial conditions Model physics parametrizations Surface parameters Lateral boundaries

41 LAM EPS Workshop, Madrid, 3-4 October 2002 Lateral Boundary Conditions Large domain designed to allow uncertainties to grow within the domain, but... –By T+72 significant uncertainty may emanate from beyond the western boundary –Error breeding will grow modes over the previous 24h, so important even for 48h forecasts

42 LAM EPS Workshop, Madrid, 3-4 October 2002 Lateral Boundary Conditions Options: ECMWF Ensemble Random perturbations Global model breeding at low resolution

43 LAM EPS Workshop, Madrid, 3-4 October 2002 Lateral Boundary Conditions ECMWF Random Global breeding Readily available –especially if use member-state time on ECMWF computers But… Possible balance problems using LBCs from different model Each new EPS run has new perturbations - no continuity with the LAM bred modes –likely generate noise

44 LAM EPS Workshop, Madrid, 3-4 October 2002 Lateral Boundary Conditions ECMWF Random Global breeding Simple to apply Usual problem of random perturbations - not focussing on the growing modes

45 LAM EPS Workshop, Madrid, 3-4 October 2002 Lateral Boundary Conditions ECMWF Random Global breeding Avoids problems of others: –identifies growing modes –continuity from run to run But… Expensive, unless run at low resolution –grid-length for LBCs should not be more than 4-5 times longer

46 LAM EPS Workshop, Madrid, 3-4 October 2002 Lateral Boundary Conditions Options: ECMWF Ensemble Random perturbations Global model breeding at low resolution No decision has been taken

47 LAM EPS Workshop, Madrid, 3-4 October 2002 Outline of LAMEPS Plans Ensemble based on European Mesoscale –20km grid-length initially –Minimum 10 members –Run to T+48, possibly to T+72 later Error breeding - possibly EnKF later Multi-schemes for convection –research into stochastic physics Perturbed Surface Roughness Perturbed LBCs –ECMWF EPS or low-resolution Global breeding

48 LAM EPS Workshop, Madrid, 3-4 October 2002 Collaboration Opportunity Dispersed multi-model ensemble Relatively simple approach to model errors Share computing demands Share system maintenance demands Option to run multiple components at ECMWF on member-states’ time

49 LAM EPS Workshop, Madrid, 3-4 October 2002 Planned Time-Scales Start work April 2003 Year 1 (incl. Relocation): –Error breeding system –Convection perturbations –First test run Year 2 ( ): –Version 1 of full perturbation system –System set up for real-time running Year 3 ( ): –Verification report on real-time performance New Met Office HQ, Exeter

50 LAM EPS Workshop, Madrid, 3-4 October 2002 Questions?