International Workshop on High Impact Weather Research, Ningbo, China, 20-23 January 2015 Evaluation of Probabilistic Precipitation Forecast Using TIGGE.

International Workshop on High Impact Weather Research, Ningbo, China, 20-23 January 2015
Evaluation of Probabilistic Precipitation Forecast Using TIGGE data over Huaihe Basin ZHAO Linna State Key Laboratory of Severe Weather Chinese Academy of Meteorological Sciences China Thank you Mr. Chairman! Good morning! Ladies and gentlemen! I will introduce you something about the TIGGE Database and its experiment on Hydrometeorologic forecasting in China. I am very happy to have an opportunity to give a talk here. TIAN Fuyou, WU Hao, QI Dan, DI Jingyue National Meteorological Center, China Meteorological Administration , China 1

Contents Introduction The data and the test catchment Results Outlook

1. Introduction Totally 81% economic losses caused by flood and drought, China 28% economic losses caused by flood 53% economic losses caused by drought In recent years, floods caused by torrential rain are the main disasters in the country, causing heavy loss of lives and property. most of heavy flood disaster is still in our memory, for exm. 我国的显著的特点：一个是损失大，气象灾害占全部自然灾害中在70％以上； Regional flood Agr. drought Flash flood

1. Introduction More than 90% provinces of China suffered from floods and drought disasters every year 1/2 of the population, 1/3 of the cultivated land and 3/4 of the industrial and agricultural output in China centralize in the middle and lower reaches of the seven major river-valleys The distribution of regions which are threaten by flood less serious regions serious regions The distribution of regions which are easy to form drought over China ——防洪抗旱减灾管理（中国气象局）水利部张志彤 half of the population, one third1/3 of the cultivated land and three-quarters3/4 of the industrial and agricultural output in China centralize in the middle and lower reaches of the 7 main rivers

55% of GDP, is the direct economic losses caused by flood in 2013
The changes of the deaths subject to the flood disaster all over the country since 2000 775 deaths by flood in 2013 55% of GDP, is the direct economic losses caused by flood in 2013 The changes of direct economic losses caused by flood all over the country since 2000 Refer to GAZETTE OF THE MINISTRY OF WATER RESOURCES OF THE PEOPLE‘S REPUBLIC OF CHINA, Aug. 2014

1. Introduction Rainfall is one of the most important weather phenomena which could result in severe flood and huge economic loss. Providing timely and accurate quantitative precipitation forecast (QPF) is a primary goal of operational prediction, especially for hydrometeorology. Probabilistic forecasting is a powerful tool to improve early warning of such highimpact events. TIGGE data provides a very good opportunity to make probabilistic forecasting of precipitation Although the ensemble prediction system (EPS) can predict the probability of occurrence of any event, and can provide more consistent successive forecasts ( Buizza, 2008). Buizza (2008) summarized two of the main advantages of ensemble-based probabilistic forecasts as that an EPS can predict the most likely scenario, the other is that an EPS can predict the probability of occurrence of any event, and can provide more consistent successive forecasts.

1. Introduction But, numerical weather forecasts without uncertainties speciﬁed are hard to be incorporated into operations and decision making of other downstream applications such as early warning of ﬂoods and rainfall induced geological hazards. To make full use of all the information available in an ensemble forecast, the main objective of this work is to give a evaluation of probabilistic precipitation with TIGGE database.

The ensemble systems used in this work
Totally 87 members The ensemble systems used in this work ECMWF NCEP CMA Country/Domain Ensemble members Forecast length Perturbation method Horizontal resolution Vertical resolution Europe 51 15 days Singular Vectors TL399 (0-10 d) TL255 (11-15 d) 62 United States 21 16 days ET (Ensemble Transform) T126 28 China 15 10 days BV (Bred Vectors) T213 31 As the forecast days are different, only 1-10 days’ predicted precipitation data are analyzed. two multi-model ensemble systems designed: MM-1: was composed with the EPS of ECMWF, NCEP and CMA MM-2: was consisted of ECMWF EPS and NCEP EPS only

The catchment of Dapoling-Wangjiaba
The dynamics of precipitation including spatial and temporal distribution is very irregular and changes from year to year. This is attributed to the basin location in the transitional area between the southern monsoon and the northern continental climate. The Huaihe River is located between latitudes 31°N and 35°N and longitudes 112ºE and 121ºE. It originates in the Tongbai Mountains of Henan Province, flows into the Yangtze River and covers four provinces, China. The length of the main channel of the Huaihe River is km and the total area of the basin is 1.912×105 km2. Its mean annual precipitation and runoff depth is approximately 888 and 240 mm respectively. Wangjiaba

Observation Stations in the Test Catchment of Dapoling-Wangjiaba
The Dapoling-Wangjiaba catchment is the origination of the Huaihe River Basin Sub-catchment locates at the upper reach of Huaihe River Basin Coverage of about 30,630 km2 (is 16% of Huaihe basin) Altitude ranging from 200 to 500 meters The time lasts from 1 July to 6 August, 2008, for totally 37 days 19 rain gauges in the test catchment The hourly accumulative rain gauge data the bilinear interpolation method was implemented in model point to rain gauge

Contents Introduction The data and the test catchment Results Summary

3. Results The probability distribution of predicted daily precipitation Brier Score Variation of ROC areas (Relative Operating Characteristics area ) Percentile A case: 23 July- 4 Aug 2008 severe precipitation event The Experiment of Probabilistic flood prediction by TIGGE database

① The probability distribution of predicted daily precipitation
-3 day -1 day Though the three single EPS show similar results, the curve of probability density function for ECMWF is close to the frequency of the observations’ daily precipitation. Both the MM-1 and MM-2 which are slanted to the left strongly show no apparent superiority compared to the three single EPS. Compared to observation, both ECMWF and NCEP slightly underestimate showers and light rain and overestimate moderate rain. CMA tends to overestimate the daily precipitation less than 8.0 mm day −1 and underestimate heavy rainfall especially heavier than 20 mm day −1 . For 3-day forecasts, ECMWF shows a similar performance to the 1-day forecasts. Both PDFs of NCEP and CMA illustrate strong skew to the left, indicating overestimates of moderate rain. The skew to the left for MM-1 and MM-2 were both greatly improved compared to the 1-day forecasts, however, both show overestimates for showers and light rain and underestimates for precipitation heavier than 4.0 mm day −1 . For 1-day forecasts, all the five EPS give high probability of predicted precipitation which are less than 8.0 mm day −1 .

The results of the 5-day forecasts
are very similar to the 3-day forecasts, except that ECMWF gives an overestimate for all precipitation less than approximately 16.0 mm day −1 . For 10-day forecasts, only ECMWF has a similar PDF to the PDF of observed precipitation albeit with overestimates for light and moderate rain and underestimates for heavier rain.

Comparing the all 1-day, 3-day, 5-day and 10- day lead times, it is clear that αˆ for ECMWF uniformly decreases from (1-day) to (10-day), while βˆ uniformly increases from mm (1-day) to mm (10-day), which is closest to the observation values as the lead time extends from 1-day to 10-day.

② Brier Score (1st Jul. - 6th Agu. 2008)
Little rain Threshold 1-10mm No rain Threshold 1.0mm Moderate rain Threshold 10-25mm Heavy rain Threshold 25-50mm Fig. 4 shows the variety and the inter-comparison of the three EPS of the four category precipitation with the brier score as the lead time ranges from 1 to 10 days. We can see from Fig. that as the lead day increases from 1 to 10, the Brier Score of the CMA no rain category forecast reduced from 0.34 to 0.28, and for MPS it decreases from 0.37 to 0.33, both are better than EC and NCEP. The Grand ensemble has the best performance for little rain forecast, and CMA does the worst, which is in agreement with the threat score and bias score 17

③ Variation of ROC areas as forecast lead days ranging from 1 to 10 days

③ Variation of ROC areas as forecast lead days ranging from 1 to 10 days
Note that except NCEP, all EPS show areas below the ROC curves greater than 0.5 for all 10 day forecasts, thus showing the ability to discriminate precipitation events. The areas under the fitted ROC curves displayed very similar behavior. Even though they were constructed with two or three of the EPS, the MM-1 and MM-2 showed no improvement compared to ECMWF with all 10 day’s forecast. MM-2 is a little superior to MM-1 as the ROC area is always higher

④ Percentile 1-day’ forecast 5-day’ forecast 10-day’ forecast CMA, EC, and NCEP display the temporal variation of the precipitation well, longer the lead time, larger the extension of the box-and-whisker plots, which indicates the increase of forecast error as the lead time ranges. In order to reveal the result of probabilistic forecast intuitively, the analyses of percentile precipitation will be shown HERE, THE FIG displays the running daily areal rainfall of observation and the prediction with lead time of 1, 5, and 10 days. daily areal rainfall of observation and the prediction with lead time of 1, 5, and 10 days.

⑤ A case: 23 July- 4 Aug 2008 severe precipitation event
Multi-M 25mm/24hr CMA 25mm/24hr ECMWF NCEP 25mm/24hr Grand 25mm/24hr Multi-M ECMWF NCEP 50mm/24hr CMA 50mm/24hr Grand 50mm/24hr 50mm/24hr Obs. 51.6mm/24hr

Spatial distribution of the 95th percentile precipitation with a 1 day’ lead time
CMA EC NCEP Xixian Wuyang GrandE CMA预报量级严重偏小，都较好的给出了降水的空间变化。 Obs: 22 Jul Jul UTC Fig. for the severe rainfall event on 23 July, 2008，with a 1-day lead time. Each EPS contributes to the MPS, either to the spatial distribution or to the precipitation intensities. Obs. for the severe rainfall event on 23 July, 2008，with a 1-day lead time. Each EPS contributes to the MPS, either to the spatial distribution or to the precipitation intensities. 22 22

two station percentile precipitation forecast with lead time ranges from 1 to 10 days.
Wuyang Station Xixian Station Fig. gives an example of 1-10 day’ forecast of Xixian station on 23 July, The observed rainfall is 51.6mm in 24 hours, which is an extreme rainfall event. We can see clearly from Fig. that the CMA tends to underestimate the precipitation intensity while the EC and NCEP give proper precipitation estimates. Obs: 22 Jul Jul UTC. the CMA tends to underestimate the precipitation intensity while the EC and NCEP give proper precipitation estimates

Ensemble forecasts of precipitation
wangjiaba CMA Xi Xian CMA The CMA-EPS largely underestimated the precipitation amount while the NCEP-EPS missed many precipitation events especially at Xixian. For most of the forecast days, the EC-EPS produced precipitation forecasts within the range of 5th–95th percentile, which are closest to the observation, so it outperformed the other two EPSs. The Grand-EPS production is the best since almost half of the forecasts are within the range of 25th–75th percentile. These results are in accordance with the results of the river discharge predictions. The performance of the ensemble precipitation forecasts plays an important role in the forecasts of the river discharge. wangjiaba NCEP Xi Xian NCEP wangjiaba ECMWF Xi Xian ECMWF Why there is difference performance of discharge prediction between xixian and Wangjiaba? Let us check the Ensemble forecasts of precipitation… wangjiaba Grand Xi Xian Grand

⑥ The Experiment of Probabilistic flood prediction by TIGGE database
TIGGE-NCEP TIGGE-CMA TIGGE-ECMWF VIC hydrological model Confluence Model Runoff Export section flow Hydrological probabilistic forecast

Results _ River discharge prediction
day Xi Xian The NCEP-EPS and CMA-EPS can bracket half of the observed discharges. The 5th–99th percentile distribution of the EC-EPS is large and nearly brackets all discharge observations during the period 23 July–3 August 2008 except for the flood ascending period on 24 and 25 July The performance of the EC-EPS is the best among the three systems, which is consistent with the precipitation forecast results of the EPSs. The performance of the Grand-EPS is equal or better than that of EC-EPS. Both the single EC-EPS and the Grand-EPS can bracket most of the observations between 5th and 99th quantile. The 5th–99th percentile distribution of the EC-EPS is large and nearly brackets all discharge observations during the period 23 July–3 August 2008 except for the flood ascending period on 24 and 25 July (Fig. 2a). The NCEP-EPS and CMA-EPS (Figs. 2b and 2c) can bracket half of the observed discharges. Thus, the performance of the EC-EPS is the best among the three systems, which is consistent with the precipitation forecast results of the EPSs (Zhao et al., 2010). The performance of the Grand-EPS (Fig. 2d) is equal or better than that of EC-EPS. Both the single EC-EPS and the Grand-EPS can bracket most of the observations between 5th and 99th quantile. 26 26

Results _ River discharge prediction
day Wang Jiaba Wangjiaba station is the outlet of the upper Huaihe River catchment. All of the EPSs predict the flood in good agreement with the observed discharge, which falls in the 5th–99th quantile except the NCEP-EPS Wangjiaba station is the outlet of the upper Huaihe River catchment. All of the EPSs predict the flood in good agreement with the observed discharge, which falls in the 5th–99th quantile except the NCEP-EPS (Fig. 4). Large basin forecasts are more skilful for 3 days lead time; Forecast error of stream-flow do not usually reduced by driven VIC 27 27

4. Outlook Based on the development of numerical prediction, the feasibility and ample space of developing probability forecast have been well documented Something we must point out is that, the bilinear interpolation method was implemented in this study, and the effect of topography was not considered. More work should be done about how to particularly set the weight of each member of the MPS. The precipitation usually affected by the total number of ensemble members, though not sure how many members are necessary.

4. Outlook Grand-EPS produces more reliable predictions of a flooding event and therefore brings significantly valuable results for the operational flood forecasting and warning service. This work gives an encouraging indication that a multi- model ensemble can provide more valuable probability forecasts than a deterministic prediction for extreme flood events. Probabilistic hydrological forecasting based on ensemble is foreseen as inevitable in the development of hydrological forecasting in future.

Thank you for listening
Refer: Zhao Linna, Fuyou Tian, Hao Wu, et al，Verification and Comparison of Probabilistic Precipitation Forecasts Using the TIGGE Data in the Upriver of Huaihe Basin，Advances in Geosciences, 2011 , 29, Zhao Linna, et al，Assessment of Probabilistic Precipitation Forecasts for the Huaihe Basin Using TIGGE Data, Meteorological Monthly (in Chinese), 2010，36（7）：

Another reason is … The different performances of the EPSs at the two stations may be caused by the differences in the geographic location and topographical distribution. Xixian station is located in the upriver of the entire basin, where the topography is fairly complex so there will be no enough time for this station to respond to the heavy rainfall. But Wangjiaba station, located at the exit cross-section of the catchment, has sufficient time to respond to the flood process

TIGGE the THORPEX Interactive Grand Global Ensemble
accelerate improvements in the accuracy of 1-day to 2-week high-impact weather forecasts. develop future Global Interactive Forecasting System (GIFS), which signifies all parts of the system from observations through data assimilation and the forecast production to the end-user applications are integrated, adaptive, and interactive. Outputs collected in near real time and stored in a common format for access by the research community (not for operational use now) Easy access to long series of data is necessary for applications such as bias correction and the optimal combination of ensembles from different sources, convenient for other applications such as probabilistic flood forecast. (Shapiro et al, 2004, Thorpex Plan)

Archive Status and Monitoring, Data Receipt
Archive Centre Current Data Provider NCAR NCEP CMC UKMO ECMWF MeteoFrance KMA CMA CPTEC IDD/LDM HTTP FTP NCDC JMA BoM

3. Methodology Threat score (TS )
The dichotomous forecast verification

三、方法强降水概率 Brier评分百分位降水
36/33 三、方法强降水概率 Brier评分百分位降水式中p是百分位，是第i百分位降水，A为升序排列的成员降水；Qi(p) is the i-th percentile areal precipitation, A is the array of the forecasted areal precipitation in ascending order ROC曲线

37/33 1. 强降水概率 Gamma分布式子中是gamma函数。估计值：

3. Methodology Areal percentile precipitation
An established percentile method presented by Hyndman and Fan (1996) is adopted for the areal percentile precipitation. The equation is given as where j = int (p × n+(1+p)/3) and γ = p × n + (1 + p)/3 − j, p is the percentile, Qi(p) is the percentile areal precipitation, A is the array of the forecasted areal precipitation in ascending order, and n is the number of ensemble members. The areal precipitation is obtained by averaging the records of 19 observations or simulated precipitation values. 38 38 38

3. Methodology BS : the score can take on values only in the range of
II. Brier Score BS : the score can take on values only in the range of 0 and 1 (inclusive). BS is small, means the prediction is better N denotes the forecast-event pairs fi is the forecast probability of a given atmospheric phenomenon or level, and Oi is the observations by considering that the observation is Oi =1 if the event occurs, and that Oi =0 if the event does not occurs. Less accurate forecasts receive higher Brier Scores, and the score can take on values only in the range of 0 and 1 (inclusive).

3. Methodology III. Percentile 5% 25% 50%(median) 75% 95%
ak small large 　统计学术语，如果将一组数据从大到小排序，并计算相应的累计百分位，则某一百分位所对应数据的值就称为这一百分位的百分位数。可表示为：一组n个观测值按数值大小排列如，处于p%位置的值称第p百分位数。　　中位数是第50百分位数。分位数是用于衡量数据的位置的量度，但它所衡量的，不一定是中心位置。百分位数提供了有关各数据项如何在最小值与最大值之间分布的信息。

Relative Operating Characteristics area (ROC area)
f(noise) f(signal) Near perfect forecast 1 Hit rate No skill forecast Real forecast 1 False alarm rate Yuejian Zhu Decision threshold

Relative Operating Characteristic (ROC) curve
Measures ability to discriminate between events and non events Measures forecast resolution conditioned on observations Plot (false alarm rate, hit rate)

4. ROC曲线 ROC面积概率预报技巧—— 相对操作特性ROC （Relative Operating Characteristic ）
44/33 概率预报技巧—— 相对操作特性ROC （Relative Operating Characteristic ）如果预报是以天气事件发生概率的形式提供给决策者，那么用户面临的问题是在什么概率阈值采取防护措施是对于阈值P*的概率预报从0变到1间的一系列取值，命中率Η 和误报率Φ 间的关系曲线为了更直观，容纳更多信息，常将命中率 f 沿着假警报率增加方向（x）积分，可以得到ROC面积AROC来描述预报系统的能力。 ROC面积命中率 ROC曲线下面的面积A可作为概率预报系统质量的指标。一个完美的预报系统的A=1.0，而没有技巧的预报系统（H=F）的A=0.5。ROC的优点之一是它允许确定性和概率预报系统间的直接比较。误报率

3. Methodology The time lasts from 1 July to 6 August, 2008, for totally 37 days. All precipitation intensities were taken into consideration with the percentile precipitation evaluation for examining the ability of forecasting the extreme rainfall events The precipitation was divided into four categories The precipitation was divided into four categories with the consistency of operational forecasts at the National Meteorological Center (NMC) of CMA as no rain, little rain, moderate rain and heavy rain with thresholds of 0.1mm (including), 9.9mm (including), 24.9mm (including) and 49.9mm (including), respectively. The very heavy rain was not taken into consideration in the threat score, bias score and brier score assessment NO rain little rain moderate rain heavy rain <1.0 1.0~10.0mm 10.0~25.0mm 25.0~50.0mm

Threat Score & Bias Score (1st Jul. - 6th Agu. 2008)
No rain: Threshold 1.0mm Threat Score Bias Score CMA is best

Little rain Threshold 1-10mm Threat Score Bias Score CMA is best

Moderate rain Threshold 10-25mm Threat Score Bias Score NCEP is best

Heavy rain Threshold 25-50mm Threat Score Bias Score ECMWF is best

4. Results (con.) the three EPSs and the Grand EPS have similar performance for the no rain forecast when the lead time is longer than 6 days. Threat score was reduced to 0.5 from about 0.6 while the bias score increase from about 1.6 to 2.0, which is reasonable according to Bermowitz and Zurndorfer (1979) NCEP is a little better as the lead time is less than 5 days. The Grand EPS has an equal or slightly better threat score than that of the EC, but much better than that of CMA and NCEP as the Grand EPS is the combination of the three EPSs.

International Workshop on High Impact Weather Research, Ningbo, China, 20-23 January 2015 Evaluation of Probabilistic Precipitation Forecast Using TIGGE.

Similar presentations

Presentation on theme: "International Workshop on High Impact Weather Research, Ningbo, China, 20-23 January 2015 Evaluation of Probabilistic Precipitation Forecast Using TIGGE."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

International Workshop on High Impact Weather Research, Ningbo, China, 20-23 January 2015 Evaluation of Probabilistic Precipitation Forecast Using TIGGE.

Similar presentations

Presentation on theme: "International Workshop on High Impact Weather Research, Ningbo, China, 20-23 January 2015 Evaluation of Probabilistic Precipitation Forecast Using TIGGE."— Presentation transcript:

Similar presentations

About project

Feedback