1. Update on the WIS Monitoring project
TT-GISC – Brasilia – October 2015
Rémy Giraud (Météo-France)
Weiging Qu (Bureau of Meteorology)
Yasutaka Hokase (Japan Meteorological Agency)

2. The timeline of the project
Jan 2014 WIS Monitoring workshop (Geneva)
Sep 2014 CBS-Ext.
Nov 2014 TT-GISC Telco
Nov 2014 Initial Draft JSON (JavaScript Object Notation) specification proposed by BoM
Dec 2014 Draft JSON specification v0.1 proposed by CMA, DWD, JMA (based on BoM)
Jan 2015 TT-GISC Telco
Mar 2015 Action being structured as a project with W.Qu (BOM) and R.Giraud (Météo-France) as leaders
CBS-2016 Submit amendment for the Manual on WIS and guidance material for the Guide onWIS
--- abstract from final report ---
The Commission welcomed the interim guidance on WIS monitoring (at that specifies near-real time and quarterly monitoring practices, and encouraged the Implementation Coordination Team on Information Systems and Services (ICT-ISS) to proceed with a pilot implementation with volunteer GISCs so that the interim guidance can be tested and updated with the aim of producing a standard practice for inclusion in WMO-No Guide to the WMO Information System at CBS‑16.
--- abstract from final report ---
The meeting agreed to organize WIS monitoring demonstration for Congress 17. It established this project as a new TT-GISC action 14/2-03.
Participants will be BoM, CMA, DWD, JMA, INMET, UK and TT-DC reps.

3. Step 1: The WIS monitoring workshop in 2014
1st step: WIS monitoring workshop in January 2014
Objectives of monitoring:
Identification of WIS monitoring and reporting requirements;
Formulation of an Implementation Strategy for the WIS Monitoring and Reporting;
Identification of clear boundaries between WIS monitoring and the monitoring that needs to be undertaken by application programmes.
Based on this the workshop has defined a set of requirements for both “real-time” and “quarterly” reporting
As part of this is was agreed to set up a pilot to assess whether the parameters to monitor and the technical solutions presented in the report were both acceptable
Report is available at:

4. Workshop outcome : Real-time monitoring
Real-time Operations Monitoring To ensure the smooth functioning of the WIS; each GISC continuously monitors in real-time items covering the GISC's area of responsibility, including:
Service availability: data distribution system, WIS catalogue and catalogue synchronization
WIS Core Network usage
Number of WIS catalogue records
Number of data and/or products stored in 24 hour cache
Number of data and products received from each Centre

5. Workshop outcome : Quarterly and incident reporting
Quarterly reporting
Service Performance;
Operational Infrastructure Performance;
Operational Anomalies and Incidents;
Evolutions/Upgrades Carried Out During the Reporting Period;
Planned Evolutions/Upgrades;
User Service Statistics
Incident reporting
To let users know any operational anomalies or incidents; each WIS centre reports events that have significant detrimental impact on either the provision of the WIS services (e.g. failure to fulfil the service requirements) or the availability of the operational infrastructure.

6. Step 2: The « technical design » of the reporting
BOM with the support of JMA, CMA and DWD has prepared a detailed technical specification for a limited set of « real-time » monitoring
This covers the type of information to monitor, the format (JSON) required to publish this information…
Example of JSON file

7. The agreed monitoring information so far
JSON over HTTP (pull request)
Three JSON files have been defined:
monitor.json
Services status (catalogue, distribution, OAI-PMH)
Snapshot of content holdings at 00 UTC (number of total metadata, data, placeholder metadata, etc.)
Accumulated daily network traffics from (number of insertion/update, deletion)
Metadata about the GISC (name, contact, etc.) and the JSON file (version, timestamp, etc.)
Addresses of other two monitoring JSON files
centres.json
Accumulated daily data/product traffics from each centres in GISC's area of responsibility
events.json
Events announcement (duration of maintenance, down time, etc.)

8. The architecture of “real-time” monitoring
GISC
DCPC NC
GISC
DCPC NC
GISC
DCPC NC
JSON
JSON
JSON over HTTP (pull request)
HTTP
JSON over
HTTP (pull)
Common
Dashboard
Aggregate JSON files
JSON
JSON
GISC
DCPC NC
Area of Responsibility
GISC
DCPC NC
The WIS Monitoring Pilot is a collective effort of all WIS Centres to provide comprehensive and up-to-date information that enable reports on system status including availability, stability, integrity.
Monitoring messages are pulled from GISC centres as JSON messages. The messages are aggregated and displayed in a web portal.

9. Step 3: the demo at WMO Congress June 2015

10. Examples of pilot dashboard
JMA’s prototype
Dashboards developers are
CMA
INMET
JMA
The prototype is available on each web page.
INMET’s prototype
CMA’s prototype

11. The outcome of the WIS monitoring demo
10 GISCs on board!
Global display is helpful to show connections for data exchange through WIS and understandable to the general public.
Time-series graph is helpful to monitor what is going on in data exchange and management. It can imply something happend when increasing/decreasing drastically, even if it does not show some reference criteria and an expected range.
Calendar showing maintenances is useful especially for WIS operators.
Further challenges (difficult?)
Using some static information from CPDB could be one of options in future work. (e.g. focal point, GTS links)
Some of the monitoring items may need to be displayed against reference criteria, which require more information to be provided by the centres. This would be further challenges which should be implemented after decision about what to monitor, though it is very difficult.  Let’s have this in the back of our head.

12. The GANTT of the project (1)

13. The GANTT of the project (2)

14. The plan for the upcoming year
After Congress (and with the summer period!), the project has unfortunately slowed down
The two project leaders are in addition in charge of the WP2 will need to coordinate further with the other GISCs is order to define additional monitoring parameters
The project leaders would very much welcome the active participation of additional GISCs in WP2 activities
A.Vocino (Italy) in liaison with TT-DC is in charge of the requirements for DCPC and NC
Once specifications for GISC (WP2) and for DCPC/NC (WP3) are ready, the work on WP4 (consolidation) will start.
A second workshop will take place toward the end of Q1-2016/start of Q to finalize the requirements and work on the guidance and implementation material
The very important deadline is CBS2016. We will need everyone on board to comply with this.

15. Recommended text TT-GISC:
Thanks the three GISCs having developed on a pilot of the WMO common dashboard for their effort in preparing such tools
Welcomes the participation of 10 GISCs during the demo at Congress
Considers that the method defined during the workshop in 2014 to publish and gather statistics using JSON file format is suitable and should be used to further enhanced the monitoring of the WIS
Recognizes the importance of monitoring the WIS to ensure suitable performances and therefore encourages all the GISCs to participate in the current project

16. Questions ?