The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Verification and Metrics (CAWCR)

Slides:

Advertisements

Similar presentations

Verification Activity in Sub-seasonal Forecasting at JMA 1 Climate Prediction Division / JMA Yuhei Takaya WWRP/THORPEX/WCRP Sub-seasonal to Seasonal Implementation.

Advertisements

ECMWF Slide 1Met Op training course – Reading, March 2004 Forecast verification: probabilistic aspects Anna Ghelli, ECMWF.

Sub-seasonal to seasonal prediction David Anderson.

1 Verification Continued… Holly C. Hartmann Department of Hydrology and Water Resources University of Arizona RFC Verification Workshop,

14 May 2001QPF Verification Workshop Verification of Probability Forecasts at Points WMO QPF Verification Workshop Prague, Czech Republic May 2001.

Summary of Progress with POAMA/ACCESS Seasonal Prediction System.

Seasonal Climate Predictability over NAME Region Jae-Kyung E. Schemm CPC/NCEP/NWS/NOAA NAME Science Working Group Meeting 5 Puerto Vallarta, Mexico Nov.

Verification of probability and ensemble forecasts

Details for Today: DATE:3 rd February 2005 BY:Mark Cresswell FOLLOWED BY:Assignment 2 briefing Evaluation of Model Performance 69EG3137 – Impacts & Models.

Seasonal Predictability in East Asian Region Targeted Training Activity: Seasonal Predictability in Tropical Regions: Research and Applications 『 East.

Gridded OCF Probabilistic Forecasting For Australia For more information please contact © Commonwealth of Australia 2011 Shaun Cooper.

© Crown copyright Met Office Forecasting the onset of the African rainy seasons Michael Vellinga, Alberto Arribas and Richard Graham S2S Conference, Washington,

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Seasonal dynamical prediction of coral.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology 3.1 Prediction skill in the Tropical Indian.

1 Intercomparison of low visibility prediction methods COST-722 (WG-i) Frédéric Atger & Thierry Bergot (Météo-France)

Seamless precipitation prediction skill in a global model: Actual versus potential skill Matthew Wheeler 1, Hongyan Zhu 1, Adam Sobel 2, and Debra Hudson.

Barbara Casati June 2009 FMI Verification of continuous predictands

Exeter 1-3 December 2010 Monthly Forecasting with Ensembles Frédéric Vitart European Centre for Medium-Range Weather Forecasts.

Introduction to Seasonal Climate Prediction Liqiang Sun International Research Institute for Climate and Society (IRI)

Caio A. S. Coelho Department of Meteorology University of Reading Met Office, Exeter (U.K.), 20 February 2006 PLAN OF TALK.

© Crown copyright Met Office An Introduction to Long-range Forecasting Emily Wallace Nov 2012.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Oscar Alves, Li Shi, Yonghong Yin, Robin.

Basic characteristics of stratospheric predictability: Results from 1-month ensemble hindcast experiments for Masakazu Taguchi Aichi University.

Verification of ensembles Courtesy of Barbara Brown Acknowledgments: Tom Hamill, Laurence Wilson, Tressa Fowler Copyright UCAR 2012, all rights reserved.

Intraseasonal TC prediction in the southern hemisphere Matthew Wheeler and John McBride Centre for Australia Weather and Climate Research A partnership.

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Data mining in the joint D- PHASE and COPS archive Mathias.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Summary/Future Re-anal.

© Crown copyright Met Office Long-range forecasting Emily Wallace Nov 2012.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Red - ECMWF Green - Sintex Navy - POAMA-2.0.

EUROBRISA Workshop – Beyond seasonal forecastingBarcelona, 14 December 2010 INSTITUT CATALÀ DE CIÈNCIES DEL CLIMA Beyond seasonal forecasting F. J. Doblas-Reyes,

Verification methods - towards a user oriented verification WG5.

EUROBRISA WORKSHOP, Paraty March 2008, ECMWF System 3 1 The ECMWF Seasonal Forecast System-3 Magdalena A. Balmaseda Franco Molteni,Tim Stockdale.

Heidke Skill Score (for deterministic categorical forecasts) Heidke score = Example: Suppose for OND 1997, rainfall forecasts are made for 15 stations.

Model validation Simon Mason Seasonal Forecasting Using the Climate Predictability Tool Bangkok, Thailand, 12 – 16 January 2015.

Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Seasonal Forecasting From DEMETER to ENSEMBLES Francisco J. Doblas-Reyes ECMWF.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Subseasonal prediction.

Eidgenössisches Departement des Innern EDI Bundesamt für Meteorologie und Klimatologie MeteoSchweiz Using seasonal forecasts to estimate current climate.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Coupled Breeding for Ensemble Multiweek.

Probabilistic Forecasting. pdfs and Histograms Probability density functions (pdfs) are unobservable. They can only be estimated. They tell us the density,

Theoretical Review of Seasonal Predictability In-Sik Kang Theoretical Review of Seasonal Predictability In-Sik Kang.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Oscar Alves and the POAMA Team CAWCR (Centre.

© Crown copyright Met Office Standard Verification System for Long-range Forecasts (SVSLRF) Richard Graham, Met Office Hadley Centre. With acknowledgements.

Standard Verification Strategies Proposal from NWS Verification Team NWS Verification Team Draft03/23/2009 These slides include notes, which can be expanded.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Sub-Seasonal Prediction Activities and.

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Understanding and predicting the contrast.

Verification of ensemble systems Chiara Marsigli ARPA-SIMC.

ECMWF Training course 26/4/2006 DRD meeting, 2 July 2004 Frederic Vitart 1 Predictability on the Monthly Timescale Frederic Vitart ECMWF, Reading, UK.

© Crown copyright Met Office Seasonal forecasting: Not just seasonal averages! Emily Wallace November 2012.

Multi Model Ensembles CTB Transition Project Team Report Suranjana Saha, EMC (chair) Huug van den Dool, CPC Arun Kumar, CPC February 2007.

Details for Today: DATE:13 th January 2005 BY:Mark Cresswell FOLLOWED BY:Practical Dynamical Forecasting 69EG3137 – Impacts & Models of Climate Change.

DOWNSCALING GLOBAL MEDIUM RANGE METEOROLOGICAL PREDICTIONS FOR FLOOD PREDICTION Nathalie Voisin, Andy W. Wood, Dennis P. Lettenmaier University of Washington,

VERIFICATION OF A DOWNSCALING SEQUENCE APPLIED TO MEDIUM RANGE METEOROLOGICAL PREDICTIONS FOR GLOBAL FLOOD PREDICTION Nathalie Voisin, Andy W. Wood and.

Verification of operational seasonal forecasts at RA-VI Regional Climate Center South East European Virtual Climate Change Centre Goran Pejanović Marija.

Verification methods - towards a user oriented verification The verification group.

Long-lead streamflow forecasts: 2. An approach based on ensemble climate forecasts Andrew W. Wood, Dennis P. Lettenmaier, Alan.F. Hamlet University of.

Caio A. S. Coelho, S. Pezzulli, M. Balmaseda (*), F. J. Doblas-Reyes (*) and D. B. Stephenson Forecast calibration and combination: A simple Bayesian approach.

International Workshop on Monthly-to-Seasonal Climate Prediction National Taiwan Normal Univ., October 2003 Evaluation of the APCN Multi-Model Ensemble.

Verifying and interpreting ensemble products

Shuhua Li and Andrew W. Robertson

Anne Leroy Météo France, Nouméa, Nouvelle-Calédonie Matthew C. Wheeler

Forecast Assimilation: A Unified Framework for the

Seasonal prediction of South Asian summer monsoon 2010: Met Office

Probabilistic forecasts

Predictability of Indian monsoon rainfall variability

Caio Coelho (Joint CBS/CCl IPET-OPSLS Co-chair) CPTEC/INPE, Brazil

COSMO-LEPS Verification

Fig. 1 The temporal correlation coefficient (TCC) skill for one-month lead DJF prediction of 2m air temperature obtained from 13 coupled models and.

GloSea4: the Met Office Seasonal Forecasting System

Short Range Ensemble Prediction System Verification over Greece

Presentation transcript:

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Verification and Metrics (CAWCR)

Through the application of appropriate verification methods we: Define the skill of predicting Australian climate on intra-seasonal timescales (forecast products) Examine the impact of initialisation; ensemble generation and model development on forecast skill Diagnose/understand mechanisms and sources of predictability Monitor in real-time Purpose – beyond forecast products

Variables - Precipitation (ensemble mean anomaly; tercile catagories) - Tmax (ensemble mean anomaly; tercile catagories) - Tmin (ensemble mean anomaly; tercile catagories) - SST (ensemble mean anomaly) - MJO RMM indices - Monsoon onset - Other key indices (SAM, blocking) Stratification - Season (DJF; MAM; JJA; SON) - According to state of ENSO/IOD/SAM/MJO (degree of stratification possible depends on sample size) Lead time and averaging period - Daily (usually done for prediction of indices) - Fortnight 1 - Fortnight 2 - Month 1 Reference forecast - Persistence of observed - Climatology Leave-one-year-out cross validation (verification of hindcasts) Comparing systems with different ensemble sizes

Probabilistic scores - Reliability (attributes ) diagram (including frequency histogram) Dichotomous (yes/no) forecasts Conditioned on the fc: given the fc, what was the outcome Reliability, resolution, sharpness Aggregation over region/seasons Need fairly large sample (enough cases in each bin) Choice of bin width - ROC curve Dichotomous (yes/no) forecasts Conditioned on obs: given the outcome, what was the fc Resolution (discrimination) Aggregation over region/seasons - ROC area Grid-point basis and showing significance: determined using the Mann–Whitney U statistic (Mason and Graham, 2002; Wilks, 2006). - Brier Skill Score (w.r.t. climatology) Debiased BSS (bias correction for small ensemble size) BBSd - based on weighting of indiv models for MME (not done this yet) Decomposition of BS: BS rel and BS res components “Deterministic” scores - Correlation of ensemble mean (usually use temporal rather than spatial) - RMSE ensemble mean - Spread of ensemble

Examples: Rainfall over Australia (Sep/Oct/Nov, fortnight 2) Reliability diagram ROC curve ROC area Brier skill score Correlation BS res ROC A BS rel