1. Romanian National Meteorological Administration | Presenter: Oana Nicola Authors: Oana Nicola, Andrei Diamandi, Alina Ristea Using ADAGUC for NWCSAF products visualization at NMA – a test case NWCSAF 2015 U SERS ’ W ORKSHOP Romanian National Meteorological Administration

2.  At National Meteorological Administration, satellite data and EUMETSAT SAF products are used in both operational weather forecasting and research. Data received through EUMETCast is further processed (e.g. by NWCSAF MSG and PPS software) and disseminated in near real time to the National and Regional Forecasting Centers and to other end users. Imagery data and satellite products are displayed in the NexReap meteorological visualization system – a quite outdated and proprietary system. A rolling image products archive of 3 months on an Intranet server is available through a web browser on the Intranet while a subset is archived on the main storage system offering ftp access.  During the last years, there has been a growing demand for satellite products in general and for NWCSAF products in particular, as well as for an easier and faster access to these data. Outline Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

3. In order to meet these demands, our group has been looking at a number of available visualization tools, which would: Be easy to implement in an operational environment. Allow visualization of data in hdf/netCDF formats (thus eliminating the need for exporting data in other formats for visualization – e.g. tiff for nexReap). Permit overlaying the displayed images with meteorological fields (e.g. NWP data). Eliminate multiple automatic/manual ftp transfers to the visualization workstation – by accessing data directly from a single dedicated server. Allow intranet /extranet access. Requirements Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

4. In this respect, the ADAGUC (Atmospheric Data Access for the Geospatial User Community) system – developed by KNMI – has been selected and tested at NMA. This presentation describes our experience installing and using ADAGUC in a test environment. http://adaguc.knmi.nl ADAGUC presentation Romanian National Meteorological Administration Atmospheric data access for the geospatial user community NWCSAF 2015 USERS’ WORKSHOP

5.  ADAGUC is a geographical information system to visualize netCDF files via the web. The software consists of a server side C++ application and a client side JavaScript application. The software provides several features to access and visualize data over the web, it uses OGC standards for data dissemination.  The spatial data infrastructure is based on OGC compliant web services. These web services include Web Mapping Services (WMS) for online visualization, Web Feature Services (WFS) for downloading vector data and Web Coverage Services (WCS) for downloading raster data. ADAGUC presentation, Cont. Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

6. Data types Vector data - Swath information (NetCDF) Grid or field information - Model/satellite/radar etc. (NetCDF) Grid or field information - ASCAT data (NetCDF) Vector components in gridded fields (NetCDF) RGBA grid information (NetCDF) Point data (NetCDF) Forecast reference times from model runs HDF5 data ADAGUC supported data formats Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

7. Tools to handle data  Generic NetCDF tools The NetCDF4 software contains two very useful tools for exploring and manipulating NetCDF data files: ncdump can show the contents (headers only or headers and data) for any NetCDF file and is very useful for inspection of NetCDF data when configuring an ADAGUC service layer. ncgen can generate a NetCDF file from an ASCII file in the CDL format (the format that ncdump dumps the data in).  GRIB data conversion tools Weather model data is most often coded in GRIB-1 or GRIB-2 format. Some tools which can convert GRIB files to NetCDF files are: grib2netcdf from ECMWF's grib_api package (in development) cdo from the Max Planck Institute for Meteorology.  Manipulating NetCDF data with Python Python has some very good possibilities for manipulating large fields of data and writing data into NetCDF4 files (netcdf4- python library). In combination with grib_api and/or the pygrib library files can be converted from GRIB to NetCDF. ADAGUC supported data organization Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

8.  As mentioned before, Adaguc can visualise a number of data types from data in NetCDF datafiles (NetCDF3 and NetCDF4). Data can also be read from some types of HDF5 files. With the great help of Maarten Plieger (KNMI) the NWCSAF MSG hdf5 image-like products were converted to the KNMI hdf5 format (structure), using the HDF5 library.  To make the work even easier for the users, a direct thumbnail button was added under “Add data” of the ADAGUC Viewer, so the NWCSAF MSG data can be displayed directly, in a specific geographic projection. ADAGUC and NWCSAF MSG products Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

9. ADAGUC Viewer DEMO version Romanian National Meteorological Administration Example: Add data / SAFNWC_MSG_prod DEMO version NWCSAF 2015 USERS’ WORKSHOP

10.  An image of a virtual machine is available and was installed to use the ADAGUC software directly. The ADAGUC image was used inside Virtualbox. The installation was straightforward and very low time consuming.  By default 1024 Mb of RAM and 2 CPU cores are assigned to the VM, this is enough to run the ADAGUC software, but we increased to 4096 Mb the amount of RAM for a better experience.  The simple step to do after starting the ADAGUC VM was to update the software, that was outdated. There are only few simple procedures in order to update the software to the latest version. ADAGUC installation – NMA test Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

11.  The server requires the following libraries: HDF5, NetCDF4, UDUNITS, GDAL, LIBXML2, GD, Cairo, Proj.4 and PostgreSQL. (already installed on the ADAGUC VM)  The environment variable ADAGUC_CONFIG must be set to the location of the configuration file. This can be done by using a bash script as CGI, in which the environment variable is set. Example script: #!/bin/bash export LD_LIBRARY_PATH=/build/lib/:$LD_LIBRARY_PATH export PROJ_LIB=/build/share/proj/ export ADAGUC_CONFIG=/config/config.xml adagucserver  The server can read single files, or multiple files varying along a dimension stored together in a directories. The server uses a database to lookup which dates belong to which file and time index. After changing the configuration, the database needs to be filled with data, this can be done with the following command: adagucserver --updatedb --config myconfig.xml Overview of the installed ADAGUC VM Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

12. NMA workflow consisted in a few very simple steps:  Copy the NWCSAF MSG products in a specific directory of the ADAGUC server: adagucserver/data  Create the configuration file (ex: cloud_mask.cgi) and copy it to: data/services/cgi- bin  The corresponding cloud_mask.xml was also created in the following dir: data/services/config  Run the command: /data/software/adagucserver/bin/adagucserver --updatedb -- config /data/services/config/nume_fisier.xml  The last step is the visualisation of the products in ADAGUC Viewer. ADAGUC Server local configuration Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

13. #!/bin/bash export LD_LIBRARY_PATH="/data/build/lib/:$LD_LIBRARY_PATH" export ADAGUC_CONFIG=/data/services/config/msg_meteo_ct.xml export ADAGUC_LOGFILE=/data/log/server.log export ADAGUC_ERRORFILE=/data/log/server.errlog export ADAGUC_DATARESTRICTION="SHOW_QUERYINFO|ALLOW_WCS|ALLOW_GFI|ALLOW_METADATA" /data/software/adagucserver/bin/adagucserver returnCode=$? if [ "$returnCode" != "0" ]; then echo "Content-type: text/html" echo "" echo "Message: " echo rv: $returnCode if [ "$returnCode" == "143" ]; then echo "= Application was killed"; fi; echo " " echo "/Message: " fi Example of ADAGUC configuration file Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

14. Example of NWCSAF Cloud Mask in ADAGUC Viewer Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

15. ADAGUC Viewer and NWCSAF MSG Cloud Mask Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

16. ADAGUC Viewer and NWCSAF MSG Cloud Type Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

17. NWCSAF MSG Convective Rainfall Rate Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

18.  ADAGUC server and viewer were easy to install and configure  Allow visualization of the NWCSAF MSG hdf5 image-like products and will be further analized for bulletin-like products. For the moment ADAGUC is not supporting the bulletin-like products, but they can still be displayed after converting to netcdf point files and then assigning symbols to plot for the values that are defined in the ADAGUC server configuration file. This part will be done with the kind help of ADAGUC team.  ADAGUC Server/Viewer eliminate multiple automatic/manual ftp transfers to the visualization workstation – positive feedback from our forecasters. Conclusions Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

19. ADAGUC Workshop  An ADAGUC workshop will be organized in June at KNMI. For further details: http://adaguc.knmi.nl Atmospheric data access for the geospatial user community Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP

20. Thank you for your attention! Romanian National Meteorological Administration NWCSAF 2015 USERS’ WORKSHOP