Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Framework of the WMO/WWRP FROST-2014 Forecast Verification Setup and Activities The Framework of the WMO/WWRP FROST-2014 Forecast.

Published byAlberta Morgan Modified over 8 years ago

Similar presentations

Presentation on theme: "The Framework of the WMO/WWRP FROST-2014 Forecast Verification Setup and Activities The Framework of the WMO/WWRP FROST-2014 Forecast."— Presentation transcript:

1 pertti.nurmi@fmi.fi The Framework of the WMO/WWRP FROST-2014 Forecast Verification Setup and Activities The Framework of the WMO/WWRP FROST-2014 Forecast Verification Setup and Activities Pertti Nurmi et al. (*) Pertti Nurmi et al. (*) (*) Acknowledgements : WWRP FROST-2014 Expert Team & FMI Verification System Development Team Other FROST-2014 Presentations in this Session (10 Sep 2015) 2.Operational Forecast Verification, by D. Kiktev et al 3.End-user Precipitation Forecast Verification, by L. Nikitina et al 4.Deterministic NWP Verification, by A. Bundel et al 5.High-impact Weather Forecasts Verification, by A. Muraviev et al 6.Spatial Verification of Precipitation Fields, by A. Bundel et al

2 pertti.nurmi@fmi.fi Forecast & Research in the Olympics Sochi Testbed - FDP & RDP - Forecast & Research in the Olympics Sochi Testbed - FDP & RDP - FROST-2014FROST-2014 FDP ~ Forecast Demonstration Project RDP ~ Research Demonstration Project 2

3 pertti.nurmi@fmi.fi Focus : High-impact Weather  Forecast Verification Here : High-impact  Specific Forecast Thresholds End-users  Olympics Participants, Audiences Outline : 1.FROST-2014 goals and general setup 2.FROST-2014 components and participants: Observations Nowcast Systems Deterministic NWP Probabilistic EPS 3.Verification FMI Verification System Some Results Impacts 4.Future visions Focus : High-impact Weather  Forecast Verification Here : High-impact  Specific Forecast Thresholds End-users  Olympics Participants, Audiences Outline : 1.FROST-2014 goals and general setup 2.FROST-2014 components and participants: Observations Nowcast Systems Deterministic NWP Probabilistic EPS 3.Verification FMI Verification System Some Results Impacts 4.Future visions More details in next talks Much more results in next talks 3

4 pertti.nurmi@fmi.fi 4 FDP ~ Forecast Demonstration Projects RDP ~ Research Demonstration Projects World Weather Research Program

5 Goals of FROST-2014 : RDP + FDP 1.To develop a comprehensive data resource of high altitude winter weather observations  Testbed 2. To exploit and improve in complex terrain winter environment : o Now-cast systems of high-impact weather phenomena - wind, precipitation type and intensity, visibility, etc. o High-resolution deterministic meso-scale forecasts o Regional, meso-scale ensemble forecasts 3.To improve the understanding of physics of high-impact weather phenomena End-users 1.To deliver deterministic and probabilistic forecasts 2.to Olympic weather forecasters and decision-makers 3.in real-time 2.To assess the benefits of forecast improvement  Verification 5

6 COSMO, Env. Canada, FMI, KMA, METN, NOAA, ZAMG + Roshydromet, IRAM supervision by WWRP WGs on Nowcasting, Mesoscale Forecasting & Forecast Verification Research COSMO, Env. Canada, FMI, KMA, METN, NOAA, ZAMG + Roshydromet, IRAM supervision by WWRP WGs on Nowcasting, Mesoscale Forecasting & Forecast Verification Research Participants of FROST-2014 6

7 FROST Nowcast Systems 6 RDP : Green / FDP : Blue FROST Deterministic Forecast Systems 5 FROST Ensemble Forecast Systems 7 7

8 pertti.nurmi@fmi.fi 8

9 9

10 XXII Olympic Winter Games 10 Some Forecast Verification Issues … #1 All modelers always verify their own fc products  General practice ! … BUT can be biased  NOT sufficient / adequate in FROST context Centralized by Roshydromet AND ”External” verification by FMI … also to “validate verification results”  enables comparisons  Due to our FMI Harmonie model runs, FMI was not “external” anymore FMI : Point Verification  User needs  Users require thresholds for their decisions  Impacts  Sports activities and events take place at (or along) points  Enhanced observation network by using 31 selected stations Formal verification period : 15 January – 15 March = 2 months FMI state-of-the-art, on-line, point verification package was “tuned” for Sochi  HUGE undertaking

11 pertti.nurmi@fmi.fi Properties & Features Web-based; Interactive; Menu-driven; 2 languages Daily forecast vs. observation comparison Monthly, seasonal and annual statistics All basic – and some novelty - verification metrics Comparison: Forecaster end-product vs. model + model vs. model Database : Gridded data + stations over Finland Sochi Olympics version XXII Olympic Winter Games FMI Operational Real-time Point Verification System 11

12 pertti.nurmi@fmi.fi XXII Olympic Winter Games Feb - Mar 2014 FMI Operational Verification System Web User Interface 12

13 pertti.nurmi@fmi.fi Verification User Interface Adapted for Sochi 2014 Olympics 13 NB : NOT used in real-time

14 pertti.nurmi@fmi.fi FMI Verification User Interface Adapted for Sochi 2014 Olympics frost-2014.fmi.fi User: sochi Pwd: d8jJ-ZbBS 14

15 pertti.nurmi@fmi.fi Valley (c. 600m) Ski Jump Ski Biathlon (Laura, top 1400m) Alpine Ski (Rosa Khutor, top 2300m) FMI Verification User Interface Adapted for Sochi 2014 Olympics 15

16 pertti.nurmi@fmi.fi XXII Olympic Winter Games Model Analysis / FC hoursContributorFMI VerificationNotes Deterministic Forecasts COSMO-RU - 7 km 00, 06, 12, 18 UTC / + 78 hrHMC,Russia *** COSMO-RU - 2 km 00, 06, 12, 18 UTC / + 42 hr ” *** COSMO-RU - 1 km - ” to be considered as RDP GEM - 2.5 km 23 UTC / + 27 hrEnv. Canada *** GEM - 1 km 21 UTC / + 25 hr ” *** GEM - 0.25 km 00 UTC / + 24 hr ” *** Processing difficulties HARMONIE-Sochi - 1 km 00, 06, 12, 18 UTC / + 36 hrFMI, Finland *** NMMB - 1 km 00, 12 UTC / + 24 hrNOAA,USA *** KMA - KMA, S-Korea to be considered as RDP ARPA (Sochi-mini) 00, 12 UTC / + 72 hrARPA SIMC, Italy *** INCA - ZAMG, Austria Joint (also now-casts) Hourly / + 48 hr“consensus” *** Ensemble Forecasts Aladin-LAEF-EPS - 11 km -ZAMG,Austria GLAMEPS - 11 km 06, 18 UTC / + 54 hrHIRLAM, Norway *** GLAMEPS - updated T Hourly / + 48 hr ” *** HarmonEPS - 2.5 km 06, 18 UTC / + 30 hr ” *** to be considered as RDP COSMO-RU-EPS - 2 km -HMC,Russia to be considered as RDP COSMO-S14-EPS - 7 km -ARPA SIMC, Italy NMMB-EPS - 7 km -NOAA, USA Nowcast systems MeteoExpert 10 minutes / + 4 hrIRAM,Russia *** INCA ZAMG,Austria INTWEnv.Canada CARDSEnv.Canada to be considered as RDP ABOM Env.Canadato be considered as RDP RW Model ( Harmonie-driven ) FMI, FinlandVerification within external CoMoSeF project FROST-2014 : FMI Verification - Models – Contributors 16

17 pertti.nurmi@fmi.fi XXII Olympic Winter Games 17 FROST-2014 : Weather variables – Potential Impact Thresholds Temperature ( O C )T < -20-20 ≤ T < -5-5 ≤ T < -2-2 ≤ T < 00 ≤ T < 22 ≤ T < 5T ≥ 5 Wind speed ( m/s )WS ≥ 3WS ≥ 4WS ≥ 5WS ≥ 7WS ≥ 11WS ≥ 15WS ≥ 19 Horizontal visibility ( m )V < 100V < 300V < 1000V < 10 000 Precipitation amountRR < 0.3RR ≥ 0.3RR ≥ 1.0RR ≥ 5.0RR ≥ 10.0RR ≥ 15.0 1-hr and 24-hr (mm) Photo © johnny9s on FlickrPhoto © Prentice Korea on Flickr High-impact  What is High-impact ?  Thresholds

18 pertti.nurmi@fmi.fi Mountain cluster Determ. models HARMONIE best during day, worst during night is the feature in the mountains COSMO worst XXII Olympic Winter Games 18 T MAE 00UTC forecast runs (available for Olympics forecasters in the morning) used in verification statistics FROST Verification : FMI examples

19 pertti.nurmi@fmi.fi XXII Olympic Winter Games 19 Determ. models GEM 0.25km and JOINT best HARMONIE ”in the middle” Lo-res model, COSMO 7km, worst Mountain cluster WS SEDI 00UTC forecast runs (available for Olympics forecasters in the morning) used in verification statistics FROST Verification : FMI examples

20 pertti.nurmi@fmi.fi EPS models Calibrated + hourly T-updated GLAMEPS best Raw GLAMEPS worst Mountain cluster NB: Actual Olympics two-week period XXII Olympic Winter Games 20 PoT ROC A All forecast runs used in the EPS verification statistics FROST Verification : FMI examples

21 pertti.nurmi@fmi.fi XXII Olympic Winter Games 21 Sochi 2014 Road Weather Pilot: End-user Feedback Scale : 1 = Fully disagree / 2 = Disagree / 3 = Neutral / 4 = Agree / 5 = Fully agree Date Friction forecast gave a good estimate of slipperiness FC storage terms (water, snow, ice) are realistic Road weather forecast was useful Notes (free text) 16.2.2014 54 Precipitation, however road surface dried towards Adler 20.2.2014 552 Dry road surface, i.e. No real need for RW forecast 21.2.2014 53 Variable road surface humidity 23.2.2014 552 Dry road surface, i.e. No real need for RW forecast End-user = Finnish skiing service team, Feedback : ” HARMONIE was superior ” ” HARMONIE visibility forecasts were excellent ” ” HARMONIE precipitation and cloud height fcs were highly useful ” ” HARMONIE five-panel user interface was really good ” ! Impacts to End-users ! FROST Verification : FMI examples

22 pertti.nurmi@fmi.fi XXII Olympic Winter Games 22 Subjective Evaluations Model Overall usefulness Forecast accuracy Visualization (appearance) Timeliness and reliability TempPrecipWindGustsVisib COSMO-RU 7 km 1.9 2.3 2.1 x2.9 COSMO-RU 2 km2.7 2.3 2.1 x2.9 COSMO-RU 1 km 1.52.0 2.3 x NMMB 1 km GEM 250 m 2.0 2.2 2.0 2.2 1.3 2.0 1.3 2.0 1.8 Harmonie 1 km 2.3 2.0 ( * ) Forecast quality assessment : 10 HMC Russia forecasters / team leaders Scale : 0 = Not useful / 1 = Partly useful / 2 = Useful / 3 = Excellent Acknowledgement : Michael Tsyrulnikov et al. HMC Russia ( * ) : Harmonie DID produce (good) visibility forecasts which were not used by Sochi forecasters ( not disseminated as XML ) FROST Verification

23 pertti.nurmi@fmi.fi 23 Re-run statistics now that full, final data sets are available  Roshydromet… Comprehensive diagnostic verification and inter-comparison of the participating forecasting systems, case studies, numerical experiments, assessments of predictability  Ongoing / Roshydromet… Open FROST-2014 data access for international research community Joint reporting-publishing-presentations by FROST-2014 Partners  WWOSC, Montreal, Aug 2014  IUGG, Prague, June 2015  EMS-ECAM, Sofia, 10 Sep 2015  Special Issue in “Russian Meteorology and Hydrology”, Aug 2015; English version issued by Allerton Press in the US  AMS Special Collection in Journal of Applied Meteorology and Climatology, Weather and Forecasting & BAMS, Feb 2016 Dedicated FROST-2014 Verification Session ! Further Steps !

24 Durban, South Africa, 6 July, 2011 RDP & FDP RDP & FDP @ 2018 Winter Olympics Thank You Questions?

Download ppt "The Framework of the WMO/WWRP FROST-2014 Forecast Verification Setup and Activities The Framework of the WMO/WWRP FROST-2014 Forecast."

Similar presentations

5 th International Conference of Mesoscale Meteor. And Typhoons, Boulder, CO 31 October 2006 National Scale Probabilistic Storm Forecasting for Aviation.

5 th International Conference of Mesoscale Meteor. And Typhoons, Boulder, CO 31 October 2006 National Scale Probabilistic Storm Forecasting for Aviation.
“A LPB demonstration project” Celeste Saulo CIMA and Dept. of Atmos. and Ocean Sciences University of Buenos Aires Argentina Christopher Cunningham Center.

“A LPB demonstration project” Celeste Saulo CIMA and Dept. of Atmos. and Ocean Sciences University of Buenos Aires Argentina Christopher Cunningham Center.
Development of Bias-Corrected Precipitation Database and Climatology for the Arctic Regions Daqing Yang, Principal Investigator Douglas L. Kane, Co-Investigator.

Development of Bias-Corrected Precipitation Database and Climatology for the Arctic Regions Daqing Yang, Principal Investigator Douglas L. Kane, Co-Investigator.
1 FROST-2014 Verification activities Finnish Meteorological Institute XXII Olympic Winter Games Feb - Mar 2014 Pertti Nurmi WMO WWRP.

1 FROST-2014 Verification activities Finnish Meteorological Institute XXII Olympic Winter Games Feb - Mar 2014 Pertti Nurmi WMO WWRP.
Panel Discussion on Corporate Volunteerism Closing Plenary, June 25, 2003 Nick Keener, CCM Director, Meteorology 2009 Summer AMS Community Meeting Aug.

Panel Discussion on Corporate Volunteerism Closing Plenary, June 25, 2003 Nick Keener, CCM Director, Meteorology 2009 Summer AMS Community Meeting Aug.
“OLYMPEX” Physical validation Precipitation estimation Hydrological applications Field Experiment Proposed for November-December 2014 4th International.

“OLYMPEX” Physical validation Precipitation estimation Hydrological applications Field Experiment Proposed for November-December th International.
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Improving COSMO-LEPS forecasts of extreme events with.

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Improving COSMO-LEPS forecasts of extreme events with.
Computer Weather Forecasts for Wind Energy Plants Khanh T. Tran AMI Environmental 206 Black Eagle Ave, Henderson, NV 89015(702)564-9186http://www.amiace.com.

Computer Weather Forecasts for Wind Energy Plants Khanh T. Tran AMI Environmental 206 Black Eagle Ave, Henderson, NV 89015(702) http://
1 Modelled Meteorology - Applicability to Well-test Flaring Assessments Environment and Energy Division Alex Schutte Science & Community Environmental.

1 Modelled Meteorology - Applicability to Well-test Flaring Assessments Environment and Energy Division Alex Schutte Science & Community Environmental.
Verification of Numerical Weather Prediction systems employed by the Australian Bureau of Meteorology over East Antarctica during the 2009-10 summer season.

Verification of Numerical Weather Prediction systems employed by the Australian Bureau of Meteorology over East Antarctica during the summer season.
Department of Meteorology and Geophysics University of Vienna since 1851 since 1365 TOWARDS AN ANALYSIS ENSEMBLE FOR NWP-MODEL VERIFICATION Manfred Dorninger,

Department of Meteorology and Geophysics University of Vienna since 1851 since 1365 TOWARDS AN ANALYSIS ENSEMBLE FOR NWP-MODEL VERIFICATION Manfred Dorninger,
4 th COPS Workshop, Hohenheim, 25 – 26 September 2006 Modeling and assimilation efforts at IPM in preparation of COPS Hans-Stefan Bauer, Matthias Grzeschik,

4 th COPS Workshop, Hohenheim, 25 – 26 September 2006 Modeling and assimilation efforts at IPM in preparation of COPS Hans-Stefan Bauer, Matthias Grzeschik,
B08 Project November 2009 Incheon, Republic of Korea The Beijing 08 Forecast Demonstration Project and Urban Prediction Needs in China LIANG Feng and WANG.

B08 Project November 2009 Incheon, Republic of Korea The Beijing 08 Forecast Demonstration Project and Urban Prediction Needs in China LIANG Feng and WANG.
COSMO GM2013.E.Astakhova,D.Alferov,A.Montani,etal The COSMO-based ensemble systems for the Sochi 2014 Winter Olympic Games: representation and use of EPS.

COSMO GM2013.E.Astakhova,D.Alferov,A.Montani,etal The COSMO-based ensemble systems for the Sochi 2014 Winter Olympic Games: representation and use of EPS.
Laurence Wilson Associate Scientist Emeritus Environment Canada Monica Bailey, Marcel Vallee and Ivan Heckmann Verification of forecasts from the 2010.

Laurence Wilson Associate Scientist Emeritus Environment Canada Monica Bailey, Marcel Vallee and Ivan Heckmann Verification of forecasts from the 2010.
REGIONAL DECISION SUPPORT SYSTEM T. Bazlova, N. Bocharnikov, V. Olenev, and A. Solonin Institute of Radar Meteorology St. – Petersburg, Russia 2010.

REGIONAL DECISION SUPPORT SYSTEM T. Bazlova, N. Bocharnikov, V. Olenev, and A. Solonin Institute of Radar Meteorology St. – Petersburg, Russia 2010.
COSMO Annual Meeting 2005 20-23 September 2005 Zurich (Switzerland) Short-Range Numerical Weather Prediction Programme.

COSMO Annual Meeting September 2005 Zurich (Switzerland) Short-Range Numerical Weather Prediction Programme.
Summary of MAP D-PHASE Strategy and Requirements MAP D-PHASE / Olympics Project Meeting 6 February 2006 Prepared by: Ron McTaggart-Cowan.

Summary of MAP D-PHASE Strategy and Requirements MAP D-PHASE / Olympics Project Meeting 6 February 2006 Prepared by: Ron McTaggart-Cowan.
Fly - Fight - Win 16 th Weather Squadron Evan Kuchera Fine Scale Models and Ensemble 16WS/WXN Template: 28 Feb 06 Air Force Weather Ensembles.

Fly - Fight - Win 16 th Weather Squadron Evan Kuchera Fine Scale Models and Ensemble 16WS/WXN Template: 28 Feb 06 Air Force Weather Ensembles.
P. Ñurmi / 15-05-2001 WWRP QPF Verification - Prague 1 Operational QPF Verification of End Products and NWP Pertti Nurmi Finnish Meteorological Institute.

P. Ñurmi / WWRP QPF Verification - Prague 1 Operational QPF Verification of End Products and NWP Pertti Nurmi Finnish Meteorological Institute.

Similar presentations

Ads by Google