1. Validation of Satellite Precipitation Estimates for Weather and Hydrological Applications Beth Ebert BMRC, Melbourne, Australia 3 rd IPWG Workshop / 3 rd APSATS, 23 October 2006, Melbourne

2. Satellite precipitation estimates -- what do we especially want to get right? Climatologists - mean bias NWP data assimilation (physical initialization) - rain location and type Hydrologists - rain volume Forecasters and emergency managers - rain location and maximum intensity

3. Short-term precipitation estimates High spatial and temporal resolution desirable Dynamic range required Motion may be important for nowcasts Can live with some bias in the estimates if it's not too great Verification data need not be quite as accurate as for climate verification Land-based rainfall generally of greater interest than ocean-based

4. Some truths about "truth" data No existing measurement system adequately captures the high spatial and temporal variability of rainfall. Errors in validation data artificially inflate errors in satellite precipitation estimates

5. Rain gauge observations AdvantagesDisadvantages True rain measurements May be unrepresentative of aerial value Verification results biased toward regions with high gauge density Most obs made once daily

6. Radar data AdvantagesDisadvantages Excellent spatial and Beamfilling, attenuation, temporal resolution overshoot, clutter, etc. Limited spatial extent TRMM PR

7. Rain gauge analyses AdvantagesDisadvantages Grid-scale quantitiesSmoothes actual rainfall Overcomes uneven values distribution of rain gauges

8. Verification strategy for satellite precipitation estimates Use (gauge-corrected) radar data for local instantaneous or very short-term estimates Use gauge or radar-gauge analysis for larger spatial and/or temporal estimates

9. Spatial verification methods Visual ("eyeball") verification Continuous statistics (RMS, correlation, bias, etc) Categorical statistics (POD, FAR, etc.) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Scale decomposition methods Entity-based methods "standard" "scientific" or "diagnostic"

10. Step 1: Visual ("eyeball") verification Visually compare maps of satellite estimates and observations Advantage: "A picture tells a thousand words…" Disadvantages: Labor intensive, not quantitative, subjective Verifies this attribute? Location Size Shape Mean value Maximum value Spatial variability Rozumalski, 2000

11. Time series of error statistics 24-hr rainfall from NRL Experimental Geostationary algorithm validated against Australian operational daily rain gauge analysis 0.25° grid boxes, tropics only

12. Continuous statistics Mean absolute error Measures average magnitude of error Root mean square error Measures error magnitude, with large errors having a greater impact than in the MAE Mean error (bias) Measures average error Correlation coefficient Measures correspondence between estimated spatial distribution and observed spatial distribution, independent of mean bias

13. Estimated yes no yes hits misses no false correct alarmsnegatives Observed Estimated Observed False alarms Hits Misses Correct negatives Categorical statistics

14. Probability of Detection False Alarm Ratio Threat score (critical success index) Equitable threat score Bias score

15. Real-time verification example 24-hr rainfall from NRL Experimental Geostationary algorithm

16. Web links for satellite precipitation validation IPWG validation of daily rainfall estimates over many regions - http://www.bom.gov.au/bmrc/SatRainVal/validation-intercomparison.html http://www.bom.gov.au/bmrc/SatRainVal/validation-intercomparison.html Climate Rainfall Data Center (monthly validation, global) - http://rain.atmos.colostate.edu/CRDC/ http://rain.atmos.colostate.edu/CRDC/ Program for the Evaluation of High Resolution Precipitation Products - http://essic.umd.edu/%7Emsapiano/PEHRPP/ http://essic.umd.edu/%7Emsapiano/PEHRPP/ Validation of 6-hourly and daily Hydro-Estimator and other geostationary estimates over US - http://www.orbit.nesdis.noaa.gov/smcd/emb/ff/validation/validation.html http://www.orbit.nesdis.noaa.gov/smcd/emb/ff/validation/validation.html TOVAS validation of daily TRMM-based estimates over US - http://disc2.nascom.nasa.gov/Giovanni/tovas/rain.ipwg.shtml http://disc2.nascom.nasa.gov/Giovanni/tovas/rain.ipwg.shtml Forecast verification – Issues, Methods, and FAQ - http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html