1. Can Ensemble Forecasts Improve the Reliability of Extreme Flood Warnings? Jörg Dietrich, Yan Wang, Michael Denhard & Andreas Schumann Institute of Hydrology, Water Resources Management and Environmental Engineering, Ruhr University Bochum Funding: German Ministry of Education and Research (BMBF), Coordination: PTJ Deutscher Wetterdienst (DWD, German National Weather Service), Offenbach

2. J. Dietrich et al., ISFD Toronto, May 2008 Outline of the presentation ▫Introduction ▫Case study: hindcasts for the Mulde river basin ▫Development of an ensemble based flood forecast scheme ▫Conclusions and future work 2

3. Uncertainties in Flood Forecasting ▫Future development of the atmosphere cannot be perfectly forecasted ▫Initial states and boundary conditions of models may be uncertain in time and space ▫Model structure may be insufficient (model and parameter uncertainty) ▫Inadequate human interaction ▫Technical problems ▫Solution for some of the data and model uncertainties: –computation of several simulations which frame uncertainty -> ensemble techniques –probabilistic instead of deterministic forecast J. Dietrich et al., ISFD Toronto, May 20083

4. Types of Ensembles ▫Single System Ensembles –Perturbation of initial and boundary conditions, different convection schemes (physically based ensembles) –Perturbation of model parameters ▫Multiple Systems Ensembles –Combination of forecasts from different models ▫Lagged Average Ensembles –Combination of actual forecasts with forecasts from earlier model runs ▫… ▫Ensembles aim at characterizing forecast uncertainty, but there will remain uncertainty about uncertainty. 4

5. J. Dietrich et al., ISFD Toronto, May 2008 Ensembles in Operational Flood Management ▫Reliability is the ability of a system to perform and maintain its functions in routine circumstances, as well as hostile or unexpected circumstances. ▫Assessment of extreme event predictions? –Model extrapolation (unobserved situation) ▫Decision rules –Can ensembles improve decisions (economy: ratio between true and false alarms, flood defence: longer lead time)? ▫Challenges in developing an ICT system –Tremendous amount of data –Computational efficiency of the models 5

6. J. Dietrich et al., ISFD Toronto, May 2008 Mulde Case Study ▫Characteristics of the river basin: –Low mountains, fast reaction to rainfall events, flash floods –Several vulnerable cities –2002: return periods up to > 500 a ▫Study area: 6200 km² ▫Operational flood forecast system - requirements: –Meso-scale resolution (headwaters with approx. 100 – 500 km² area) –Short to very short lead times –Support decisions about flood alerts/pre-alerts Grimma, 2002-08-13. Source: dpa 6

7. Study Area – Chemnitz Sub-catchment J. Dietrich et al., ISFD Toronto, May 20087

8. Operational Ensemble Systems Used ▫COSMO-LEPS –Single system physically based ensemble, 16 members –Medium range (132 h lead time) –Meso-scale (10 km horizontal resolution) ▫SRNWP-PEPS –Multiple systems ensemble, 23 members (17 cover Mulde area) –Short range (48 h lead time) –Meso-scale (7 km horizontal resolution) ▫COSMO-DE –Deterministic model, lagged average ensemble: 7 members –Very short range (21 h lead time) –Local scale (2.8 km horizontal resolution, resolving convection) J. Dietrich et al., ISFD Toronto, May 20088 Molteni et al., 2001 Denhard and Trepte, 2006 Steppeler et al., 2003

9. Hindcasts with Raw Ensembles (2002-2006) ▫Comparison of different ensemble prediction systems ▫Aim of study: development of a scheme for adaptive combination of ensembles from different sources and with different lead times ▫Hydrological model: calibrated, assumed as perfect ▫True alerts: –2002-08: extreme flood, underestimated –2006-02/03 flood caused by rainfall/snowmelt, overestimated ▫False alerts: –2005-07, 2005-08: meteorology (no flood alert issued) –2006-08: rainfall true but overestimated, low soil moisture ▫Missings: –rainfall: not investigated, flood (T > 2 y, meso-scale): none J. Dietrich et al., ISFD Toronto, May 20089

10. 2002 Flood: COSMO-LEPS Hindcast +5 d J. Dietrich et al., ISFD Toronto, May 2008 Aug 08 th Aug 09 th Aug 10 th Aug 11 th 10

11. 2002 Flood: COSMO-DE Hindcast +21 h J. Dietrich et al., ISFD Toronto, May 2008 coloured: early good performers 11

12. 2006 False Alert: COSMO-LEPS J. Dietrich et al., ISFD Toronto, May 200812 ▫Synoptic forecast: up to 290 mm rainfall within 3 days ▫Water release from reservoir initiated ▫80 mm within 36 hrs, low soil moisture, peak discharge T < 2 y

13. Hindcasts: Alarm Level Exceedance EPSCOSMO-DE + 21h (Dt=1h)SRNWP-PEPS + 48 h (Dt=1h)COSMO-LEPS + 132 h (Dt=3h) InitializationmaxObs+21maxDetmaxMedLAFmaxObs+48maxMedmaxQ75maxObs+132maxMedmaxQ75 07.08.200273.514.547.3 08.08.200287.111.418.2 09.08.200287.127.559.3 10.08.20027.310.58.287.1124.9156.7 11.08.200226.841.820.887.1108.5195.1 12.08.200289.984.856.387.1112.1115.8 05.07.20053.33.74.93.18.223.5 06.07.20051.32.1 0.710.930.0 07.07.20050.71.6 0.711.126.4 08.07.20050.71.54.00.64.58.4 09.07.20050.42.63.30.410.018.0 29.07.20053.40.41.24.43.918.9 30.07.20053.44.85.34.47.615.7 31.07.20050.52.8 4.42.713.2 01.08.20050.41.3 4.811.917.0 02.08.20054.88.512.45.413.134.9 03.08.20054.813.415.25.416.2 04.08.20052.711.9 5.49.612.7 05.08.20058.28.19.65.413.846.8 06.08.20058.211.314.15.416.419.9 07.08.20056.08.59.75.412.016.0 03.08.20060.10.90.1 0.20.38.126.649.2 04.08.20060.11.70.2 12.0137.08.116.230.9 05.08.20069.615.52.69.613.081.58.17.012.8 06.08.20069.321.86.29.616.8136.28.120.728.1 Alarm 1 Alarm 2Alarm 3Alarm 4 13.325.643.270.8 J. Dietrich et al., ISFD Toronto, May 2008 discharge, m³/s 13

14. Lessons learnt from Hindcasts ▫COSMO-LEPS shows best performance at +2 to +3 days lead time, but often a large spread -> meteorological uncertainty high compared to hydrological uncertainty ▫COSMO-DE tends to under predict rainfall at certain model runs -> solution: lagged average ensemble, physical ensemble is scheduled for 2010 ▫SRNWP-PEPS performs well, but has outliers -> solution: plausibility check, calibration ▫We need more hindcasts to improve probabilistic assessment and to develop decision rules! J. Dietrich et al., ISFD Toronto, May 200814

15. J. Dietrich et al., ISFD Toronto, May 2008 observations radar, rain gauges Ensemble Combination - Meteorology 2007 global prediction systems meso-scale ensembles COSMO-LEPS SRNWP-PEPS deterministic local model COSMO-DE Lagged Average- Ensemble (LAF) assimilation observations radar, rain gauges 2006 2002 2005 probabilistic weather scenario: multi-model ensemble from PEPS, COSMO-LEPS, COSMO-DE model average, m approx. 10 calibration, Bayesian Model Average (BMA) assimilation 15

16. Ensemble Calibration with BMA ▫Bayesian Model Averaging assigns weights to ensemble members based on training period ▫Daily recalibration: 12 of 19 members have significant weights, 3 best members > 50%, overfitting possible J. Dietrich et al., ISFD Toronto, May 2008 Nov 1 st – 14 th 2006 Mulde catchment COSMO-LEPS median (F19) SRNWP-PEPS (F2-F18) COSMO-DE (F1) accumulated relative weight day 16 BMA further reading: J. McLean Sloughter, Adrian E. Raftery and Tilmann Gneiting: Probabilistic Quantitative Precipitation Forecasting Using Bayesian Model Averaging. Technical Report, Department of Statistics, University of Washington

17. Hydrological Modelling System ArcEGMO ▫(Semi-)Distributed, GIS-based rainfall-runoff model ▫Modular system combinig several conceptual sub-models J. Dietrich et al., ISFD Toronto, May 200817 2. Runoff concentration C1, CC1, C2, CC2: storage coefficients S1, S2: storage capacity 1. Runoff generation 3. Channel routing 1 2 3 HSC: total input HMX: input dynamic GNX: hydraulic conductivity Edited from Becker et al., 2002 -> 5 sensitive parameters for flood modelling

18. Calibration and Testing – Würschnitz/Chemnitz ▫30 flood events from 1954 – 2006, 2 y < T < 250 y ▫6 – 24 1h-stations, disaggregation of approx. 60 1d- stations (nearest neighbour) J. Dietrich et al., ISFD Toronto, May 2008 1978/051994/03 1997/07 V C 2002/08 1998/11 1996/07 18

19. J. Dietrich et al., ISFD Toronto, May 2008 Ensemble Generation - Hydrology observations flood routing/inundation models probabilistic runoff scenario for the headwaters assimilation parameter ensemble ArcEGMO training period historic flood events preconditions event type inference sequential ensemble update 12-24 hrly comp.3 hrly comp. 5d(1d) 2d(12h) 21h(3h) COSMO-LEPSSRNWP-PEPSCOSMO-DE LAF deterministic hydrological modelling ArcEGMO 19 Probabilistic weather scenario

20. Hydrological Parameter Ensembles ▫Analysis of historic flood events ▫Stable parameters for slow reacting runoff components ▫Parameters for fast reacting runoff components (mainly infiltration rate resp. generation of surface runoff) are subject of uncertainty –Problem: overlay of data uncertainty and parameter uncertainty in calibration (sp./temp. resolution of high rainfall intensities!) ▫A priori generation of sets of efficient parameters –Monte-Carlo simulation with restricted parameter ranges –Classification of flood events (rainfall intensity, antecedent precipitation) ▫Simulation with a small subset of efficient parameters –-> physically based hydrological ensemble (single model) J. Dietrich et al., ISFD Toronto, May 200820

21. Update of Ensemble Weighting (Hydrology) ▫Bayesian update of parameter ensembles based on data assimilation ▫Update of weights, not re-calibration of parameters! J. Dietrich et al., ISFD Toronto, May 200821 yellow line: observed discharge blue line: model average light blue: uncertainty band (Q95-Q5) forecast relative weight discharge

22. J. Dietrich et al., ISFD Toronto, May 2008 Conclusions and Outlook ▫Ensemble forecasts can be an integral part of an operational flood forecast system. ▫Ensembles can, but not necessarily must improve flood forecasts. ▫Limited resources require adaptive strategies for the operational application of a probabilistic flood prediction chain. ▫Further work: –Ensemble calibration using empirical orthogonal functions (Denhard et al. in prep.) –Near real-time updating of the hydrological ensembles using assimilated observed data –Analysis of 2007 – 2008 forecasts: improve basis for decision rules 22

23. J. Dietrich et al., ISFD Toronto, May 2008 Thank you very much for your attention! ▫Can Ensemble Forecasts Improve the Reliability of Extreme Flood Warnings? ▫Contact: Joerg.Dietrich@rub.de ▫Acknowledgements: Flood Management and Reservoir Authorities of Saxonia, BAH Berlin, DHI-WASY Dresden 23

24. ensemble mean [mm] [%] P >50mm SRNWP-PEPS Verification 24h precipitation, 1.5 d lead time run: 22.08.2005 0 UTC available: 22.08.2005 6:05 UTC valid: 22.08.- 23.08.2005 6 UTC 24J. Dietrich et al., ISFD Toronto, May 2008

25. Ensemble Calibration with BMA ▫Bayesian Model Averaging assigns weights to ensemble members based on training period J. Dietrich et al., ISFD Toronto, May 200825 J. McLean Sloughter, Adrian E. Raftery and Tilmann Gneiting: Probabilistic Quantitative Precipitation Forecasting Using Bayesian Model Averaging. Technical Report, Department of Statistics, University of Washington

26. J. Dietrich et al., ISFD Toronto, May 2008 Probabilistic Forecast of 2002 Flood (Reanalysis) 26

27. Forecast Reliability ▫Reliability is the ability of a person or system to perform and maintain its functions in routine circumstances, as well as hostile or unexpected circumstances. ▫The IEEE defines it as "... the ability of a system or component to perform its required functions under stated conditions for a specified period of time.“ ▫When reliability is considered from the perspective of the consumer of a technology or service, actual reliability measures may differ dramatically from perceived reliability. One bad experience can be magnified in the mind of the customer, inflating the perceived unreliability of the product. (Wikipedia) J. Dietrich et al., ISFD Toronto, May 200827