Near-Real-Time Data Collection at WDCA: Why & What

Slides:



Advertisements
Similar presentations
World Meteorological Organization Working together in weather, climate and water WMO Commission for Atmospheric Sciences: Quality Management Framework.
Advertisements

BADC Workshop 2: BADC Services to Data Suppliers Royal Met. Soc. Conference – 14 September 2005 Ag Stephens et al.
Copyright Alan Rowley Associates Steps to an Accurate Result Select a method and validate it as suitable for the purpose envisaged. Establish that.
Short Course on Introduction to Meteorological Instrumentation and Observations Techniques QA and QC Procedures Short Course on Introduction to Meteorological.
WMO WIGOS from the CAS Perspective Sandro Fuzzi National Research Council, Italy WMO: Research Dept.
An Introduction to Grants.Gov Office of Sponsored Programs Virginia Commonwealth University Spring 2006.
ResCom AUTH Research Committee - 1 Secretariat of Research Committee Aristotle University of Thessaloniki IT Department Project Management and Monitoring.
Data management in the field Ari Haukijärvi 2nd EHES training seminar.
AGENDA for RSG meeting June 4, Welcoming. Introduction of participants 2. Adoption of the agenda 3. Activities since the RSG meeting in Antalya.
CryoNet Network of Cryospheric Surface Observations Wolfgang Schöner Central Institute of Meteorology and Geodynamics Vienna, Austria.
CAPACITY Operational Atmospheric Chemistry Monitoring Missions CAPACITY Final Meeting - WP Ground Segment synthesis Final Meeting ESTEC02/06/05.
IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal MINCyT,
BSRN XI meeting 2010/04 GAW-PFR Network and its Calibration Christoph Wehrli and Stephan Nyeki Physikalisch-Meteorologisches Observatorium Davos, Switzerland.
Where Should the GALION Data Reside? Centrally or Distributed? Introduction to the Discussion Fiebig, M.; Fahre Vik, A. Norwegian Institute for Air Research.
Climate data past and future: Can we more effectively monitor and understand our changing climate? Peter Thorne.
NOAA/NESDIS/National Oceanographic Data Center Following the Flow of Two Underway Data Streams Within the U. S. National Oceanographic Data Center Steven.
Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.
Meteorological Observatory Lindenberg – Richard Assmann Observatory (2010) GCOS Reference Upper Air Network Holger Vömel Meteorological Observatory Lindenberg.
1 SUZAKU HUG 12-13April, 2006 Suzaku archive Lorella Angelini/HEASARC.
SOLAS and the British Atmospheric Data Centre Charles Kilburn Anne De Rudder.
Scenario use cases Szymon Mueller PSNC. Agenda 1.General description of experiment use case. 2.Detailed description of use cases: 1.Preparation for observation.
QA4EO in 10 Minutes! A presentation to the 10 th GHRSST Science Team Meeting.
GSICS Data and Products Servers Volker Gärtner (EUMETSAT) For GSICS Data Working Group.
GSICS GDWG Report, Toulouse, 9 February GSICS Data Working Group Status Report V. Gaertner CMA, EUMETSAT, JMA, KMA, NOAA/NESDIS, WMO.
Marianne König, Tim Hewison, Peter Miu
Common interoperability, best practices and strategic approach
Increasing the number of NRT stations and instruments
Ann Mari Fjæraa Philipp Schneider Tove Svendby
New Developments in ACTRIS Surface In-Situ Data Flow and Handling
IWDW-WAQS Technical Committee
Project monitoring and evaluation
Aerosols, Clouds and Trace gases Research Infrastructure
GSICS Collaboration Servers a Vehicle for International Collaboration Status 2011 Peter Miu EUMETSAT.
Report on “what are the emerging research needs for CAMS”
EMEP monitoring – quality issues
ACTRIS on the ESFRI Roadmap
How user feedback and requirements drive service evolution
EMODnet - Ingestion and safe-keeping of marine data
Making Data Providers’ Contribution Count
Planned Activities of GSICS Microwave sub-group
Data Flows in ACTRIS: Considerations for Planning the Future
ALICE Monitoring
SNPP VIIRS Aerosol Product Latencies
TIGGE Archives and Access
Active Data Management in Space 20m DG
SowiDataNet - A User-Driven Repository for Data Sharing and Centralizing Research Data from the Social and Economic Sciences in Germany Monika Linne, 30.
Software Requirements
Exploitation of ISS Scientific data - sustainability
Institutional role in supporting open access, open science, open data
The WMO Rolling Review of Requirements and the OSCAR tools
Ocean Prediction Networking
Computing Infrastructure for DAQ, DM and SC
Textbook Engineering Web Applications by Sven Casteleyn et. al. Springer Note: (Electronic version is available online) These slides are designed.
Cover page.
(VIP-EDC) Point 6 of the agenda
Update on the GSICS Procedure for Product Acceptance (GPPA)
Data flow ACTRIS-2.
Prepared by: Jennifer Saleem Arrigo, Program Manager
ESS.VIP VALIDATION An ESS.VIP project for mutual benefits
ETS – Air Data submission Training
Hot summary of the Fourth GSICS Users Workshop
Availability to Observation data. Kjetil Tørseth, NILU
ACTRIS – EMEP, THE WAY FORWARD
ISO 9001 – 2008 Changes Summary of Changes
Reportnet 3.0 Database Feasibility Study – Approach
Reporting QA Measures to EBAS – and a Word on Flagging
EMEP TFMM, Larnaca, 13 may 2010 Kjetil Tørseth & Wenche Aas
Wenche Aas, Kjetil Tørseth, Cathrine Lund Myhre
Introduction – workshop on EBAS and Data Quality
Benefits of ACTRIS for Implementation of EMEP monitoring strategy
Presentation transcript:

Near-Real-Time Data Collection at WDCA: Why & What Fiebig, M. et al. WDCA at the Norwegian Institute for Air Research

Source: Copernicus.eu

Implementing the Digital Agenda: Copernicus Source: Alessandro Annoni, JRC

Source: Vincent-Henri Peuch, ECMWF

Source: Vincent-Henri Peuch, ECMWF

Source: Vincent-Henri Peuch, ECMWF

New Operational Atmospheric Composition Air Quality Products CAMS (Copernicus Atmosphere Monitoring Service) AOD product:

New Operational Atmospheric Composition Air Quality Products CAMS (Copernicus Atmosphere Monitoring Service) PM2.5 forecast product: GAW surface in situ aerosol data for validation, not assimilation. Data needs to be available in NRT!!! Delayed validation for academic purposes only Non-NRT data has much less impact on result.

Source: Astrid-Christina Koch, Copernicus

Source: Astrid-Christina Koch, Copernicus

Metrological Traceability: Surface In Situ Data’s Unique Selling Point “property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty” (Definition of metrological traceability according to JCGM200:2012 International vocabulary of metrology – Basic and general concepts and associated terms) Many (most?) surface in situ aerosol observations are traceable already, and more will be. Traceable data are the only means to discover systematic biases in an absolute sense. Motivation: Hugely increased visibility, both for GAW and for station. Good arguments for sustained funding.

GAW Aerosol’s NRT Network: Current Status 32 stations 52 instruments

GAW Aerosol’s NRT Network: Possible Status 75 stations 110 instruments

Measures to Realise NRT Potential in GAW Aerosol Convince people: Station PIs need to be convinced that additional visibility is worth the extra effort in maintaining NRT data stream. NRT data delivery is an investment in a station’s future. Provide Tools for Participating: NOAA software package ACTRIS modular DAQ software Ensure Visibility of Station and Network: Means of acknowledgement in operational products not defined yet. Resistance of operational service providers to acknowledge data sources.

NRT DAQ Software Packages NOAA software package: Comprehensive, covers numerous instruments (ECOTECH & TSI neph, aethalometer, MAAP, PSAP, CPCs), acquisition, online display, QA workflow, normal & NRT reporting. ACTRIS: Volunteers write stand-alone software for single instruments to share Can be adapted to local workflow. Instrument Responsible TSI 3563 nephelometer NILU (ready!) ECOTECH nephelometer U. Crete MAAP TROPOS Magee AE31 NILU Magee AE33 Radiance Res. PSAP TROPOS SMPS Custom DMPS / SMPS Stations themselves

Measures to Realise NRT Potential in GAW Aerosol Convince people: Station PIs need to be convinced that additional visibility is worth the extra effort in maintaining NRT data stream. NRT data delivery is an investment in a station’s future. Provide Tools for Participating: NOAA software package ACTRIS modular DAQ software Ensure Visibility of Station and Network: Means of acknowledgement in operational products not defined yet. Resistance of operational service providers to acknowledge data sources.

Operational Services: What’s In It For The Station? Success for ACTRIS-ESFRI: ACTRIS-ESFRI will establish ACTRIS (stations & facilities) as permanent infrastructure. Will be successful only if relevant (for digital agenda / Copernicus) Funding & Visibility at national level: Your national funding agency can demonstrate they are contributing to Copernicus (with your station) Increased chances of getting national funding on ACTRIS ticket. Visibility: Work towards visibility of network and station in final product.

2 Routes For Data Submission Regular, annual submission: Submission of annual dataset by 30 June the following year. Data shall have highest level of QA possible. Dataset assembled by data originator, because detail knowledge resides here. Near-Real-Time Data Submission: Submission shall be collected and disseminated within max. 3 hours of measurement. Compromise on data quality due to auto-processing accepted. Alert service for provider if instrument “misbehaves”. Central data processing to ensure uniform dataset and implement alert service.

NRT data collection flow chart automatic feedback Sub-network data centre: auto-creates hourly data files (level 0). initiates auto-upload to NRT server. FTP transfer to data centre Station: collects raw data in custom format transfer Data Centre: check for correct data format (level 0). check whether data stays within specified boun- daries (sanity check). FTP transfer to data centre Station: auto-creates hourly data files (level 0). initiates auto-upload to NRT server. automatic feedback User access (restricted) via web-interface: ebas.nilu.no Processing to level 1.5 Processing to level 1 EBAS database User access via machine-to-machine web-service Hourly level 1 data file Hourly level 1.5 data file

Increasing the number of NRT stations and instruments Follow-up on stations that have NRT upload procedures ready. Take one instrument type at a time, focus first on nephelometers, then absorption photometers, then MPSS. Within ACTRIS WP3, we should provide & share DAQ software that: collects data from instrument displays incoming data graphically, writes data to hard drive in level 0 format uploads hourly level 0 files to NRT account at NILU. auto-start of the software, e.g. after a power failure. Need volunteers to write, provide, and maintain such software!!!