1. Tom Hopson, NCAR (among others) Satya Priya, World Bank
Flood forecasting precipitation products as part of the project: Development of Flood Forecasting for the Ganges and the Brahmaputra Basins using satellite based precipitation, ensemble weather forecasts, and remotely-sensed river widths and height
Tom Hopson, NCAR (among others)
Satya Priya, World Bank

2. Outline Observed Precipitation products
Ensemble Precipitation Forecasts from Global Forecast Systems
Biases and Correlations amongst the Products

3. Satellite Products
Satellite products are available as soon as each 24-hour accumulation period is completed.
Product
Name
Institution
Country
Sensor Types
Resolution
TRMM
NASA
USA
Passive microwave,
Infrared
0.25 deg
GSMAP
JAXA
Japan
0.1 deg
CMORPH
NOAA
~0.25 deg
Our NCAR merged product is a simple average of the available satellite products

4. Measurement of Precipitation -- Rain Gauges
Rain gauge estimates: NOAA CPC and WMO GTS
0.5 X 0.5 spatial resolution; 24h temporal resolution
approximately 100 gauges reporting over the Ganges-Brahmaputra
24hr reporting delay

5. Comparison of Precipitation Products:
TRMM
Note: TRMM and CMORPH use same satellite information, but different estimation algorithms

6. Movie of one day’s “merged” rainfall we are using for this project – blended precipitation estimates from NOAA CMORPH, NASA 3B4RT, and JAXA to reduce overall error. If the individual products were compared, there are clear differences in locations and intensities of rainfall at finer scales, although the general large-scale features are generally in common.

7. Outline Observed Precipitation products
Ensemble Precipitation Forecasts from Global Forecast Systems
Biases and Correlations amongst the Products

8. TIGGE Forecasts
Forecasts are on 2 day delay from TIGGE (The International Grand Global Ensemble).
Forecast Center
Country /
Region
# of Ensemble Members
Forecast
Out to:
Currently
on Display
ECMWF
Europe 50
15 days
Yes
UKMO UK 11
7 days
CMC
Canada 20
16 days
NCEP
USA
< Dec 2015
CMA
China 14 No
CPTEC
Brazil
MeteoFrance
France 34
4.5 days
JMA
Japan 26
11 days
BoM
Australia 32
10 days
KMA
Korea 23
10.5 days
Originally a project of THORPEX: a World Weather Research Programme project to accelerate the improvements in the accuracy of 1-day to 2-week high-impact weather forecasts.

9. Unique Datasets/Software Created
Thorpex-Tigge
UKMO
CMC
CMA
ECMWF
MeteoFrance
NCAR
NCEP
JMA
NCDC
IDD/LDM
HTTP
Using the Thorpex-Tigge data for this project. Green dots show forecasting centers which we are using their forecasts for.
FTP
Archive Centre
CPTEC
Current Data Provider

10. Archive Status and Monitoring, Variability between providers

11. Early May 2011, floods in southwestern Africa

12. Early May 2011, floods in southwestern Africa
-- examine ens forecasts … ECMWF 24hr precip

13. Early May 2011, floods in southwestern Africa
-- examine ens forecasts … NCEP GEFS 24hr precip

14. Early May 2011, floods in southwestern Africa
-- examine ens forecasts … ECMWF 5-day precip

15. Early May 2011, floods in southwestern Africa
-- examine ens forecasts … NCEP GEFS 5day precip

16. Movie: ECMWF 24-hr forecasted daily rainfall over the Ganges and Brahmaputra basins showing the variability in all 50 ensemble members

17. Outline Observed Precipitation products
Ensemble Precipitation Forecasts from Global Forecast Systems
Biases and Correlations amongst the Products

18. Emily Riddle, Tom Hopson
Bias and skill in rainfall forecasts over the Ganges and Brahmaputra river basins
Emily Riddle, Tom Hopson
Acknowledgements: Jenn Boehnert, Kevin Sampson, Will Cheng, Dan Collins, Dorita Rostkier-Edelstein

19. Bias in rainfall forecasts for the Ganges-Brahmaputra basin
Observed August Rainfall ( )
Weather forecasts show large regional biases in rainfall over these basins
The models vary in their spatial representation of climatological rainfall, including interactions between rainfall and topographic features.
These results highlight the importance of calibration and bias correction for these forecasts.
August Average of 0-4 day lead forecasts ( )
US NCEP Model
Canadian Model
UK Met Office Model
European Model

20. Goals:
Compare the seasonal cycles of rainfall in satellite observations and medium range forecasts from four numerical weather prediction (NWP) models over GBM catchments.
Assess the skill of these models as a function of lead time (0-14 days).

21. Spatial and temporal scales:
28 Catchments
4696 Catchments
Temporal:
24 hour accumulations: 0 to 10 day lead times
5 day accumulations: 0-4 day, 5-9 day, day lead times

22. Assessing bias: Methodology
Calculate 5-year climatologies from merged satellite precip ( )
Calculate 5-year lead-based climatologies from the 4 NWP models ( ).
Smooth climatologies with 11 day running average and regrid to catchments using correspondence files.
Calculate bias as the difference between the model and satellite climatologies.
A few caveats:
Satellite precip has errors
Using control run rather than ensemble mean for this analysis
Relatively few years, so smoothing does not eliminate all interannual variability

23. August Average of 0-4 day lead forecasts (2011-2015)
Satellite August Rainfall ( )
August Average of 0-4 day lead forecasts ( )
mm/day
NCEP Model (USA)
Canadian Model 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
UK Met Office Model
ECMWF (European)

24. August Forecasts versus merged satellite precip (2011 – 2015)
CANADA
NCEP
ECMWF
UKMET
Bias (mm/day)

25. Yamuna
Ghagra
Middle Ganges
Manas
---- ECMWF
---- NCEP
---- CMC
---- UKMET
---- OBS
Middle
Brahamaputra
Chambal
Sone
Meghna

26. Assessing forecast skill: Methodology
Make climatologies from merged satellite precip (TRMM, CMORPH, JAXA) for
Calculate lead-based climatologies from Tigge forecasts (ECMWF, NCEP, UKMET, Canada) for
Calculate daily and 5-day precipitation anomalies for each lead time and the satellite product by subtracting off the lead-based climatologies.
Calculate correlations between the anomaly time series for each catchment.

27. Correlations for JJA (2011-2015) averaged over all catchments
Daily rainfall (lines) and Pentad rainfall (x’s)
---- ECMWF
---- NCEP
---- CMC
---- UKMET
X’s: 5-day averages
Lines: 24 hour averages

28. Spatial pattern of skill: JJA, 0-4 day forecast
NCEP
CANADA
UKMET
ECMWF
Anomaly
Correlations

29. Skill as a function of catchment area: JJA 0-4 day forecast
Yarlung-
Tsangpo
Basin
(Tibet)

30. Conclusions:
Precipitation forecasts show large regional biases in rainfall, highlight the importance of calibration and bias correction.
The models vary in their spatial representation of climatological rainfall, indicating differing interactions between rainfall and topographic features.
Models with the smallest biases are not necessarily the ones with the best skill.
The relationship between forecast skill and basin size is apparent in these data, though other geographic factors are clearly important as well.

31. Additional plots showing intercomparisons between observational products and forecasts

32. 25th (bottom bar) 50th (circle) and 75th (top bar) percentile rainfall for observed and forecast rainfall products averaged over all catchments. 25th (blue) and 75th (red) percentiles are also shown for individual catchments for values that exceed / fall below the catchment averaged values.
JAXA
TRMM
GAUGE
NCEP
UKMET
CANADA
Multi-Model
Mean
MERGED
ECMWF
Persistence

33. Intercorrelations between Observations and 24-hour forecasts for July and August
Larger scale catchments
Observations
Forecasts
GAUGE
+ MERGED
Multi-Model
Mean
Persistence
ECMWF
CANADA
MERGED
NCEP
UKMET
JAXA
TRMM
GAUGE
MERGED
JAXA
Observations
TRMM
GAUGE
GAUGE + MERGED
ECMWF
NCEP
CANADA
Forecasts
UKMET
Multi-Model
Mean
Persistence

34. Intercorrelations between Observations and 24-hour forecasts for July and August
Smaller scale catchments
Observations
Forecasts
GAUGE
+ MERGED
Multi-Model
Mean
Persistence
ECMWF
CANADA
MERGED
NCEP
UKMET
JAXA
TRMM
GAUGE
MERGED
JAXA
Observations
TRMM
GAUGE
GAUGE + MERGED
ECMWF
NCEP
CANADA
Forecasts
UKMET
Multi-Model
Mean
Persistence

35. Hit Rate and False Alarm Rate for 75th percentile precipitation events averaged over all catchments. Forecasts are for 0-24h precipitation.
False Alarm Rate:
% of observed average/low precip days that were incorrectly predicted to be high precip events
(Best forecast: 0%, null forecast: 25%)
Hit Rate:
% of observed high precip events that were correctly forecast (Best forecast: 100%, null forecast: 25%)
Verification product
Gauge
Merged Sat
JAXA
TRMM
Merged + Gauge
NCEP
ECMWF
CANADA
UKMET
Multi-Model
Mean
Persistence
NCEP
ECMWF
CANADA
UKMET
Multi-Model
Mean
Persistence
Forecast Product

36. Spatial Scale Dependence: 0-24h forecasts
July and August
Uses multi-model mean for forecast
Uses multi-product mean for observations
Blue: Large scale catchments
Red: Smaller scale catchments

37. Similar plots for 5-day averages

38. Intercorrelations between Observations and 5-day forecasts for July and August
Larger scale catchments
Observations
Forecasts
GAUGE
+ MERGED
Multi-Model
Mean
Persistence
ECMWF
CANADA
MERGED
NCEP
UKMET
JAXA
TRMM
GAUGE
MERGED
JAXA
Observations
TRMM
GAUGE
GAUGE + MERGED
ECMWF
NCEP
CANADA
Forecasts
UKMET
Multi-Model
Mean
Persistence

39. Intercorrelations between Observations and 5-day forecasts for July and August
Smaller scale catchments
Observations
Forecasts
GAUGE
+ MERGED
Multi-Model
Mean
Persistence
ECMWF
CANADA
MERGED
NCEP
UKMET
JAXA
TRMM
GAUGE
MERGED
JAXA
Observations
TRMM
GAUGE
GAUGE + MERGED
ECMWF
NCEP
CANADA
Forecasts
UKMET
Multi-Model
Mean
Persistence

40. Spatial scale dependence: 0 – 5 day forecasts:
July and August
Uses multi-model mean for forecast
Uses multi-product mean for observations
Blue: Large scale catchments
Red: Smaller scale catchments

41. Observations’ Take-Home Points
satellite precip products internally highly correlated (no surprise) – however, correlation dependence on spatial scale still does imply some independent information.
This “independent information” of the satellite products is seen by the increase (albeit slight compared to TRMM) in the highest satellite product-gauge correlation occurring for the satellite “merged” product.
comparison with rain gauge: see increase in correlation both spatial (small to large catchment) and temporal (1-day to 5-day accumulations) scale also implies the skill of these products increase with scale as well, implying their utility increases as catchments increase in size.
Highest correlations of the NWP with the gauge+merged product does also imply that both gauges and satellite products provide independent information.

42. NWP Take-Home Points
significant skill increase over persistence, both small to large spatial and temporal scales – forecast information provides utility beyond just a good obs network can provide from a weather and hydrologic standpoint
although arguably Canada had some of the lowest biases, seeing may have some of the lowest skill as well.
Similarly, NCEP (most similar to NCMRWF operational model) also shows lower skill
As with satellite/gauge intra- and intercomparisions, see increase in skill of all products with spatial and temporal scales.
ECMWF coming out ahead of all 4 products (although they also seems to show some of the worst biases, interestingly enough).

43. NWP Take-Home Points (cont)
Models themselves not that highly correlated.
Multi-model (straight average) does show improvement over ECMWF although only slight
Highest correlations observed with the multi-model highlights that the lack of correlation amongst the models means that their errors actually do at least partially off-set each other.
Over-biases in all NWP (including multi-model) products relative to (almost) all “obs” products.
Note: the cross-correlation of the different “observation” products (i.e. JAXA, TRMM, gauge) with the “merged-gauge” product is approximately 0.85, which provides an estimate on the upper bound on the possible correlation the NWP TIGGE products can achieve with the “merged-gauge” product. Therefore, the multi-model mean correlation of approximately 0.73 is, in fact, approaching this upper bound.

44. “I have a very strong feeling that science exists to serve human welfare. It’s wonderful to have the opportunity given us by society to do basic research, but in return, we have a very important moral responsibility to apply that research to benefiting humanity.” Dr. Walter Orr Roberts (NCAR founder)
“I have a very strong feeling that science exists to serve human welfare. It’s wonderful to have the opportunity given us by society to do basic research, but in return, we have a very important moral responsibility to apply that research to benefiting humanity.” Walter Orr Roberts