1. S2S sub-project on verification (and products)
Objectives:
Recommend verification metrics and datasets for assessing forecast quality of S2S forecasts
Provide guidance for a potential centralized effort for comparing forecast quality of different S2S forecast systems, including the comparison of multi-model and individual forecast systems and consider linkages with users and applications
Issues to be addressed:
Identification of current practises in sub-seasonal to seasonal forecasts
Identification of user-relevant variables and quantities to be verified
Provision of guidance on minimum hindcast standards (hindcast length and ensemble size)
Promotion of subseasonal forecasting intercomparison efforts and evaluation of benefit of multi-model approach
Links with WMO Joint Working Group on Forecast Verification Research (JWGFVR), WMO CBS/CCl ET-OPSLS, WMO LC-LRFMME and other S2S sub-projects
Sub-project members: Caio Coelho, Andy Robertson, Yuhei Takaya, Richard Graham, Debra Hudson, Joanne Robbins, Angel Muñoz

2. S2S sub-project on verification (and products)
Science plan developed with inputs from JWGFVR, SERA and S2S: Wiki page: Chanel for sub-project activities dissemination
Guidance document to stimulate scientific community to address S2S verification problems

3. Sub-project activities
List of published literature on verification methods of relevance to S2S verification posted on the sub-project wiki page including:
Books and technical reports
Scientific papers

4. Sub-project activities
Literature survey on S2S verification posted on the sub-project wiki page
List of published literature on S2S verification addressing the following topics:
Assessment of S2S systems forecast skill
Assessment of MJO/ISO forecast skill
Assessment of monsoon systems forecast skill and associated characteristics
Applications
Seamless verification

5. Sub-project activities
Collaboration bwt S2S and WMO: Questionnaire on subseasonal verification practices in operational centers (GPC) prepared, with the purpose of sharing current practices used to verify subseasonal forecasts (both for operations and research) and also help identify gaps and guide novel developments. Summary of responses discussed with WMO CBS/CCl ET-OPSLS and posted on the sub-project wiki page
Development of pilot real-time sub-seasonal MME predictions: WMO LCLRFMME developed a pilot system for real-time multi-model subseasonal forecasts using real-time forecasts (and hindcasts) from a subset of models contributing to the S2S project accessible via ECMWF data archive. The S2S wiki page provides a link for the S2S research community to see the initial developments and provide feedback for future developments and improvements in this pilot under development system.
Coordination of input for WMO ET-OPSLS on S2S application areas under development and operational needs and posted on wiki page

6. Sub-project activities
Following suggestion of S2S SG-LG meeting (Jeju 2015) a comprehensive collection of links to available datasets for S2S verification posted on the sub-project wiki page
Links to reference datasets (Reanalysis and gridded products) for assessing S2S forecast quality
Atmospheric parameters (e.g. geopotential height, temperature, SLP, wind, etc)
Oceanic parameters
Surface parameters
Datasets accessible via the KNMI Climate Explorer
including links for data access via the IRI data library

7. Sub-project activities
Links and instructions on how to access S2S project model datasets posted on wiki
Summary table of S2S project models
Links for accessing S2S models data at ECMWF and CMA data portals
Links to instructions on how to extract S2S models data from ECMWF S2S portal
Link to IRI Data Library for accessing subset of these data (ECMWF, NCEP and CMA models): various file formats including OpenDAP access, including link to an introduction to the IRI Data Library
Links to S2S forecast products webpages developed by ECMWF and S2S Museum (University of Tsukuba/Oxford) posted on wiki

8. Sub-project activities
S2S included in the JWGFVR user-oriented verification challenge
Organizing S2S verification session at the 7th International Verification Methods Workshop (IVMW) in coordination with JWGFVR, Berlin,8-11 May 2017, preceded by verification tutorial, 3-6 May First announcement for abstract submission already circulated to S2S SG-LG mailing list. Workshop website will soon be available.
Key-note talk by S2S co-chair (Frederic Vitart) on verification methods and procedures used in sub-seasonal verification

9. Sub-project activities
Preparing forecast verification chapter for S2S book co-edited by S2S co-chairs
Proposed chapter outline
1) Introduction
Definition of what is meant by forecast verification
Explanation of why forecasts need to be verified
Definition of forecast goodness
Attributes of forecast quality
Touch on what makes S2S forecast verification different from verification at other time scales
2) Identification of verification goals
Designing verification questions to be addressed
Choosing verification measures/graphics
3) Observational references
Recognizing uncertainties in observational datasets
Matching forecasts and observations
4) Review of most common verification measures
Deterministic measures
Probabilistic measures
5) Types of S2S forecasts and current verification practices
Literature review of what has been done so far in S2S verification
S2S verification examples
6) Summary, recommendations and challenges in S2S verification
Chapter authors: Caio Coelho, Barbara Brown,
Laurie Wilson, Marion Mittermaier,
Barbara Casati (JWGFVR members)