1. WIGOS Data Quality Monitoring System Pilot project for the Quality Monitoring Component WebEx meeting, 1 July 2015, 2 PM (GE) WIGOS Project Office & Observing Systems Division

2. WMO Agenda 1- Review of the outcomes of the WIGOS Workshop on Data Quality Monitoring and Incident Management and Cg-17 on the WIGOS Pre- operational Phase – SG/DL; 2- The Quality Monitoring component of the WDQMS: Status, actions and proposals of a prototype for the Monitoring Pilot project – LN/TP; 3- The Incident Management component of the WDQMS: Status and actions of the Incident Management Pilot project – SG/DL; 4- Tour de table followed by discussion 5- Conclusions and Actions/next steps

3. 1- Outcomes of the WIGOS Workshop on Data Quality Monitoring and Incident Management and Cg-17 on the WIGOS Pre-operational Phase Roadmap and action items from WIGOS QM WS… 5.GOS QM Pilots (SG-QM)  Pilot (2015 - ECMWF, NCEP, others?) [R.Grumbine, D.Richardson]  Participating NWP centers to agree on:  Information to exchange;  Format for exchange;  Feasiblity and commencement. 3

4. WIGOS Data Quality Monitoring System (for the WIGOS Pre-Operational Phase 2016-18) Real-time monitoring of performance (data availability and quality) of all WIGOS components, searchable by region, country, station type, period, updated regularly will allows us to monitor regional and national (actually station-level) performance of all WIGOS components, and analyse trends over time Pilot project with ECMWF and NCEP is underway for GOS (every 6h) Delayed mode monitoring of data quality as measured against reference sources of information. Fault management system for tracking and mitigation of performance issues. Resources to develop and implement this and/or operational partnership(s) will be needed 4

5. With WDQMS, WMO can monitor and document (every 6h) what is being provided and analyze trends over time

6. Extract of «WIGOS» Resolution 4.2.2(1)/1 approved on Cg-17 The highest priorities for the WIGOS Pre-operational Phase (2016-19) will be: (a)National WIGOS implementation; (b)WIGOS Regulatory Material complemented with necessary guidance material to assist Members with the implementation of the WIGOS technical regulations; (c)Further development of the WIGOS Information Resource (WIR), with special emphasis on the operational deployment of the OSCAR databases; (d)Development and implementation of the WIGOS Data Quality Monitoring System; (e)Concept development and initial establishment of Regional WIGOS Centres; 6

7. Regional WIGOS Centers (WRC) (for the WIGOS Pre-Operational Phase 2016-18) General role of RWC would be to monitor, coordinate and support the implementation activities in the Region/sub-Region Specific areas of responsibility may be any or all of the following: Regional performance monitoring of WIGOS networksRegional performance monitoring of WIGOS networks (data availability). Regional monitoring of observational data quality, especially for observations not used for global NWPRegional monitoring of observational data quality, especially for observations not used for global NWP (climate, atm.composition, hydrol.) Fault management, i.e. follow-up with data providers in case of data availability or data quality problemsFault management, i.e. follow-up with data providers in case of data availability or data quality problems. Regional network design. Regional coordination of WIGOS implementation activities and projects. Liaise with other WMO regional Centres (RIC, RTC, RCC). 7

8. Actions/Milestones for the Monitoring pilot project (1) Develop mock-up (Timo, June) - pressure on 31 May 2015, as per ECMWF - compare with some later date Conclusion/ Next steps: a) … b) 8

9. Actions/Milestones for the Monitoring pilot project (2) Find suitable baseline for NWP monitoring data (Luis, July): Some thoughts/questions: - Baseline refers only to the nr. of stations, not to observing/reporting schedule! - Vol.A/OSCAR is the reference list of all observing stations, - WIGOS Metadata field «3-09/Station status» may be used to identify stations not exchanging data internationally (this is a mandatory field for phase 1 (2016) of the WIGOS Metadata implementation); - Possible approach is to combine various baselines together: i) Vol.A reference (all stations) = “baseline X” AND ii) [Vol.A]-[3-09/Station status] = “baseline Y” AND iii) average number of reports received over a certain time AND iv) no baseline: just the number of reports received in real-time These are all useful to trigger incident management events, even iii) and iv) Examples of visual tools to be implemented in the Quality Monitoring System: Daily (or weekly/monthly) maps of stations, with colors, using these “baselines”: For data availability status (for each variable): - nr. of received reports: 90-100% = blue, 45-90% = green, 1-45% = yellow, 0%(Y stations) = red, 0%(X stations) = brown; 9

10. Actions/Milestones for the Monitoring pilot project (2) cont. For data availability evolution (for each variable): - Increased/decreased nr. of received reports in the “aggregation period”: same color scale but with different symbols  /  For data quality (for each variable): - nr. of received and used reports: 90-100% used = dark blue, 45-90% used = blue, 1-45% used = light blue, 0 used(Y stations)=pink, 0 used(X stations)=brown; 0 used(blacklisted stations)=black Visual tools should also include graphics for individual stations with the time evolution of availability and quality. Automatic reports should be produced, per station and per country, with the number of received, used and blacklisted stations. Conclusions/Next steps: a) … b) 10

11. Actions/Milestones for the Monitoring pilot project (3) Clarify Interface with WIGOS Incident Management System - What reports are needed for the IMS? Conclusion/ Next steps: a) … b) 11

12. Actions/Milestones for the Monitoring pilot project (4) Clarify Interface with OSCAR (Timo, Etienne) - Vol.A and «other» info will be retrieved from OSCAR - some monitoring info will be passed into OSCAR Conclusion/ Next steps: a) … b) 12

13. Actions/Milestones for the Monitoring pilot project (5) Consult about involvement of other NWP centres (Luis, July/August) Conclusion/ Next steps: a) … b) 13

14. Actions/Milestones for the Monitoring pilot project (6) Specify time-resolution of the outputs (Lars Peter, Luis, Timo) - 6h, day, week, month, Conclusion/ Next steps: a) … b) 14

15. Actions/Milestones for the Monitoring pilot project (7) Specify aggregation criteria (location vs station id) (Luis, Timo, July) - Ex. a station has moved to a different location Conclusion/ Next steps: a) … b) 15

16. Actions/Milestones for the Monitoring pilot project (8) Relationship with SMM/AGM/SAM (Steve, LPR July) Conclusion/ Next steps: a) … b) 16

17. Actions/Milestones for the Monitoring pilot project (9) Specify operational requirements (technology stack, space) (Timo, Luis, November) Conclusion/ Next steps: a) … b) 17

18. Final remarks / Next meeting Final remarks … … … … Next meeting …

19. www.wmo.int Thank you for your attention www.wmo.int/wigos