Database: submission, status, future. Send data and receive email confirmation is not enough!!!! Remember to import or discard your variables!!

Slides:



Advertisements
Similar presentations
New FLUXDATA collection Aim of the presentation: get feedbacks and suggestions from you in the next days.
Advertisements

Assessment of the impacts of biofuel crops on the carbon, water, and nitrogen cycles in Central Illinois Marcelo Zeri, Andrew VanLoocke, George Hickman,
Uncertainty in eddy covariance datasets Dario Papale, Markus Reichstein, Antje Moffat, Ankur Desai, many others..
Management impacts on the C balance in agricultural ecosystems Jean-François Soussana 1 Martin Wattenbach 2, Pete Smith 2 1. INRA, Clermont-Ferrand, France.
MATLAB Extras Week 16 – 5/12/09 Kate Musgrave
Software Frameworks for Acquisition and Control European PhD – 2009 Horácio Fernandes.
Deborah Agarwal BWC technical team 16 July Applications of eddy covariance measurements, Part 1: Lecture on Analyzing and Interpreting CO2 Flux.
The Project AH Computing. Functional Requirements  What the product must do!  Examples attractive welcome screen all options available as clickable.
Raw data Hz HH data submitted for synthesis Flux calculation, raw data filtering Additional filtering for footprint or instrument malfunctioning.
Open Source Workshop1 IBM Software Group Working with Apache Tuscany A Hands-On Workshop Luciano Resende Haleh.
Fluxdata.org FLUXNET Dataset Synthesis Support Deb Agarwal (LBNL) Catharine van Ingen (Microsoft) Fluxdata Team: Marty Humphrey (UVa), Norm Beekwilder.
The FP7 How to submit a project electronically AN INFORMATION POINT FOR FP7 IN PALESTINE: Training Seminar of experts Nicosia, Cyprus November.
with contributions of 40+ research groups
LGA Consortium Meeting Sacramento, CA Web-Based Time Survey Solution Presented by Dan Gardner July 8, 2010.
Unit of Biosystem Physics 1. O BJECTIVES To analyze impacts of grazing on carbon dioxide (CO 2 ) fluxes (F) exchanged by a meadow. 2. E XPERIMENTAL SITE.
Meta-analysis of eddy covariance carbon fluxes data Dario Papale, Markus Reichstein, Riccardo Valentini Marc Aubinet, Christian Bernhofer, Alessandro Cescatti,
New FLUXDATA collection Aim of the presentation: get feedbacks and suggestions from you in the next days.
Christian Beer, CE-IP Crete 2006 Mean annual GPP of Europe derived from its water balance Christian Beer 1, Markus Reichstein 1, Philippe Ciais 2, Graham.
Deb Agarwal abd Marty Humphrey e Norman Beekwilder e Monte Goode abd
CS 114 – Class 02 Topics  Computer programs  Using the compiler Assignments  Read pages for Thursday.  We will go to the lab on Thursday.
FLUXNET and Remote Sensing Open Workshop Mathias Goeckede, Natascha Kljun, Laura Chasmer Topic 3: Fluxes, Hyperspectral Remote Sensing and Upscaling.
Long term weather and flux data: treatment of discontinuous data. Bart Kruijt, Wilma Jans, Cor Jacobs, Eddy Moors Loobos.
Inter-comparison and Validation Task Team Breakout discussion.
CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Monitoring for the LHC experiments Irina Sidorova (CERN, JINR) on.
Flux tower representativeness >croplands< news from the workbench Martin Wattenbach Astley Hastings Pete Smith.
Remote Sensing Break-out: Wed. Jan 24, 9:30-12:00 Summary report Joost van Haren Jeff Morisette.
Seasonal and Inter-Annual Variability in Net Ecosystem CO 2 Exchange at Six Forest Flux Sites in Japan Y. Ohtani* 1, Y. Yasuda* 1, Y. Mizoguchi* 1, T.
Translation to the New TCO Panel Beverly Law Prof. Global Change Forest Science Science Chair, AmeriFlux Network Oregon State University.
ICT Techniques Processing Information. Data validation Validation is the checking of data before processing takes place. You need to ensure this is done.
Gap filling of eddy fluxes with artificial neural networks
Data Release Requirements, decisions and plans. What we need New machine Implement (and test) all proposed changes from current Provide data quality summary,
Ecosystem Component – Activity 1.1 Submission status Ancillary data Data policy Software.
Ecosystem component Levi, Finland November.
Downloading and Installing Autodesk Inventor Professional 2015 This is a 4 step process 1.Register with the Autodesk Student Community 2.Downloading the.
AmeriFlux and FLUXNET Report Dennis Baldocchi Bev Law AsiaFlux Workshop, 2008, Seoul, Korea.
Free-Air Carbon Dioxide Enrichment – FACE BERAC review of DOE-funded experiments (’07) –ORNL FACE –Duke Forest FACE (FACTS I) –Rhinelander Aspen FACE (FACTS.
Terrestrial Carbon Observations TCO Previous Strategy 1- better identify the potential end users, and their requirements 2- organize and coordinate reliable.
By The Supreme Team CMPT 275 Assignment 2 May 29, 2009.
Site-Level Model-Data Comparison A Proposed NACP Interim Synthesis Project Ken Davis, Peter Thornton, Kevin Schaefer, Dan Riciutto Coordinators.
COS PIPELINE CDR Jim Rose July 23, 2001OPUS Science Data Processing Space Telescope Science Institute 1 of 12 Science Data Processing
GdI/ICS 1 WS 2009/2010 Telecooperation/RBG Prof. Dr. Max Mühlhäuser Dr. Guido Rößling Dr. Dirk Schnelle-Walka, Stefan Radomski.
CSC 480 Software Engineering Test Planning. Test Cases and Test Plans A test case is an explicit set of instructions designed to detect a particular class.
Using AmeriFlux Observations in the NACP Site-level Interim Synthesis Kevin Schaefer NACP Site Synthesis Team Flux Tower PIs Modeling Teams.
CARBOEUROPE-IP, Poznan, October 2007 ADVISORY PANEL COMMENTS by Roger Francey and Neil Turner The identification of achievements and highlights has been.
LBA Flux Tower Workshop Software Intercomparison Celso von Randow Ganabathula Prasad CPTEC/INPE Celso von Randow Ganabathula Prasad CPTEC/INPE December.
Ecosystems component – Cropland Activity Pete Smith & Martin Wattenbach Professor of Soils & Global Change School of Biological Sciences, University of.
Gap Filling Comparison Workshop, September 18-20, 2006, Jena, Germany Corinna Rebmann Olaf Kolle Max-Planck-Institute for Biogeochemistry Jena, Germany.
Thomas Foken, Mathias Göckede, Matthias Mauder University of Bayreuth, Germany Department of Micrometeorology 3 nd CarboEurope-IP Integrated Project Meeting.
FTS monitoring work WLCG service reliability workshop November 2007 Alexander Uzhinskiy Andrey Nechaevskiy.
FLUXNET synthesis: the data 932 site years from 246 sites (i.e. 50% more than in February) Available to colleagues involved in proposals at
IN Measurements 10 Hz data of 3D wind speed T sonic H 2 O CO 2 Pressure OUT Fluxes Hz data of Fc (carbon flux) LE (water flux) Hs (sensible heat.
1 09/2003 Processing Library Update CF Checker – Script made available as a web based form on the BADC site -
GLOSS Training Workshop Course Japan Meteorological Agency May 15-26, 2006 Sea Level Data Processing with SLPR2 1. Introduction.
RESERVA ECOLÓGICA DO IBGE RECOR SITE BRASÍLIA. IMAGEM DO SATÉLITE IKONOS, 2001 RECOR CAMPO SUJO CERRADO.
The CarboeuropeIP Ecosystem Component Database: data processing and availability Dario Papale, Markus Reichstein.
16-1 PC-HYSPLIT WORKSHOP Workshop Agenda Introduction to HYSPLIT Introduction.ppt Model Overview Model_Overview.ppt Meteorological Data Meteorological_Data.ppt.
WG2A meeting 7-8 October 2004 Working Group 2A ECOSTAT Agenda item 9b Discussion on final Intercalibation register.
Collecting Copyright Transfers and Disclosures via Editorial Manager™ -- Editorial Office Guide 2015.
TOPSpro Special Topics I: Database Managemen t. Agenda for Module I: Database Management  TOPSpro Backup/Restore Wizard  TOPS-TOPS Import/Export Wizard.
The information systems lifecycle Far more boring than you ever dreamed possible!
Advanced Higher Computing Science
GCE Software Systems Development
Impact of dissolved organic carbon (DOC) deposition on soil solution DOC Intern(ation)al data evaluation Arne Verstraeten ICP Forests combined Expert Panel.
Modelling the Management and Dynamics of
Patrick C. McGuire (U. Reading, NCAS, Pier Luigi Vidale (U
NACP Site MDC “Are the various measurement and modeling estimates of carbon fluxes consistent with each other - and if not, why?” Simulated vs. observed.
General recommendations
This is a marked test which will count toward the final course mark
Presentation transcript:

Database: submission, status, future

Send data and receive confirmation is not enough!!!! Remember to import or discard your variables!!

Status Looking to who submitted what…..

Status Main sites Status Main sites – LEVEL4 prod.

Associated Main sites

Status of General information and Soil Respiration files submission Soil RespGen file Soil RespGen file

In the next weeks the Greengrass database will be incorporated in the Carboeurope IP database and processed. We asked also the data from Carbomont without answer We are waiting the data from the Siberian projects (from Eurasia-net to TCOS) We will try also to obtain data from the Amazonian project from EU Any other dataset is welcome We invite also the PIs to send their data and in particular the 2003 missed months!!! NO (only partially)… Still without (partially)… Coming

Questions/discussion about the status?

Gapfilling intercomparison results

From the Gapfilling WS: Do we need more variables? Flag indicating if there are footprint problems (particularly for crop) Flags indicating the management LAI or NDVI or Vegetation height (crop and grass) NOTE! It is not a good solution to put simply qcFc = 2 because the qcFc are defined from Spoleto!

The database role in standardization 1)Storage correction whit Sc provided by the PI (NEE_or) and calculated from the CO 2 concentration top of tower (NEE_st) 2) Ustar threshold calculated centrally 3) Spike filtering performed centrally 4) Gapfilling performed centrally using the two best methods from the gapfilling intercomparison 5) Partitioning in GPP and TER performed centrally BUT What about “standard quality flags obtained from the raw data”? What about the corrections/checks applied to the raw data? What about the corrections implemented in your software? Unfortunately not all the PIs are sending the correct quality flags. Many sites have all the data with qc=0, or with all qc=2. There are sites that are not applying the qc correctly and mix-up with ustar filtering or footprint How many of you provided these info correctly to the database? Are these information important?

Setting Comparison BA – 3D – ConstTLag LD – 3D – AutTLag BA LD

µmolCO 2 m -2 s -1 Generally this has a small effect to the sums However can have effect on the single half hour

How to solve this problem? How to avoid these differences? Using the same software with exactly the same settings BUT We compared different software and they give very similar results with the same settings Using the exactly the same settings BUT Can we define a “right” set of corrections and settings? What about the uniqueness of the sites? Use what you want but tell me what you did… BUT How can you send these information to the database? How many of you sent the information about the 2D 3D or PF correction applied? How many of you are sending the updated versions of the GEN file?

The best would be to have a common software where: -The user could decide which correction to apply -There would be implemented all the possible corrections -The user could also propose and add new options/corrections -All the settings used would be summarized in a table as metadata -The output would be directly formatted to be uploaded in the DB But to work, this software should be: -As flexible as possible -Available for all the HW/OS platforms -Continuously updated -User friendly for not expert users And in particular: -FREE -OPEN SOURCE -COMMON PROPERTY (CarboeuropeIP, Fluxnet)

Are you interested in developing a common “community owned eddy covariance software”? We started developing the structure of such a software and now it is ready for your input… code: FORTRAN 77/90 OPEN-SOURCE CODE SET-UP BASED ON “.ini”-files

ACQUISITION PROCESSING core: FORTRAN 77/90 Features a user-friendly GUI Simple but less flexible GUI: C (LabWindows) Graphical version available

RAW DATA SPECTRAL ANALYSIS PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u *, T *, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging CE-IP STANDARD OUTPUT.med FILES selected datasets all raw data TRANSLATOR OTHER RAW DATA

time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u *, T *, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging Moncrieff et al., 1997 – Journal of Idrology Aubinet et al., 2000 – Advances in Eological Research Moncrieff et al, 2004 – Handbook of Micrometeorology Lee et al. 2004, Handbook of Micrometeorology Aubinet et al., 2000 – Advances in Eological Research Liu et al., 2001 – Boundary Layer Meteorolgy Webb et al., 1980 – Quart. J. R. Soc. Moncrieff et al., 1997 – Journal of Idrology Horst, 1997 – Boundary Layer Meteorology Aubinet et al., 2001 – Agricultural and Forest Meteorology Foken and Wichura, 1996 – Agricultural and Forest Meteorology Foken et al., 2004 – Handbook of Micrometeorology Thomas and Foken, 2002 – Am. Meteorol. Soc. warm thanks to the Eddy Covariance CE-IP Summer School (Wépion, July 10-21, 2006) organizers and speakers!!

RAW DATA SPECTRAL ANALYSIS PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u *, T *, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging.med FILES selected datasets all raw data TRANSLATOR OTHER RAW DATA CE-IP STANDARD OUTPUT

RAW DATA SPECTRAL ANALYSIS PLANAR FITRAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u*, T*, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging CE-IP STANDARD OUTPUT.med FILES selected datasets all raw data headers, anemometer format, scaling factors, offsets, binary … TRANSLATOR suitable format ACQUISITION SOFTWARE

RAW DATA SPECTRAL ANALYSIS PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u *, T *, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging.med FILES selected datasets all raw data TRANSLATOR OTHER RAW DATA CE-IP STANDARD OUTPUT

fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u *, T *, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging RAW DATA SPECTRAL ANALYSIS PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING.med FILES selected datasets all raw data spike removal (*) amplitude resolution (*) drop-outs (*) absolute limits (*) discontinuities (*) Skewness and Kurtosis (*) unrealistic time-lag (*) angle of attack (*) non-steady horizontal wind (*).med FILES MODIFIED RAW DATA time-lag detection and data alignment TRANSLATOR OTHER RAW DATA (*) Vickers and Mahrt, 1997 – Jour. Atm. Ocean. Tech.

RAW DATA PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u *, T *, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging.med FILES selected datasets all raw data TRANSLATOR OTHER RAW DATA SPECTRAL ANALYSIS CE-IP STANDARD OUTPUT

Quality Check / Quality Flagging Schotanus and WPL correction RAW DATA PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Fluxes, u*, T*, z/L etc.. calculation CE-IP STANDARD OUTPUT.med FILES selected datasets all raw data Calculation of scalars / w co-spetra Calculation sens. heat / w co-spectrum Calculation of exper. transfer functions automatic or manual regression with a given function Spectral correction SPECTRAL ANALYSIS If user wants to use the or spectral correction options..

RAW DATA SPECTRAL ANALYSIS PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Axis rotation Schotanus and WPL correction Fluxes, u *, T *, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging.med FILES selected datasets all raw data TRANSLATOR OTHER RAW DATA CE-IP STANDARD OUTPUT

TRANSLATOR OTHER RAW DATA RAW DATA SPECTRAL ANALYSIS PLANAR FIT ACQUISITION SOFTWARE RAW DATA SCREENING time-lag detection and data alignment fluctuation calculation mean, variances, co-variances Schotanus and WPL correction Fluxes, u*, T*, z/L etc.. calculation Spectral correction Quality Check / Quality Flagging CIP STANDARD OUTPUT.med FILES selected datasets all raw data Axis rotation passes plane coefficients to main prog for rotation angle calculation calculates planar fit (bi-linear regression) of w vs u/v If user wants to use the option..

Some preliminary validation With EDIRE BA 3D With MASE LD 3D

SOME CORRECTIONS, TESTS, OPTIONS ARE NOT YET IMPLEMENTED (EX: OGIVE TEST, FOOTPRINT TEST…) SOME ANEMOMETER AND IRGA ARE ALSO NOT YET SUPPORTED THE MODULE TO IMPORT RAW DATA FORM OTHER SOFTWARE READS NOW ONLY FEW DATA-TYPES THE SW STILL NEEDS A THOROUGH VALIDATION THE AVAILABLE GUI PRESENTLY RUNS ONLY ON WINDOWS (2000 and later).. AND SOME BUGS ARE TO BE FIXED!! WHERE WE ARE AND WHAT WE NEED.. THE SOFTWARE CURRENTLY SUPPORTS MANY OF THE FEATURES AND PROCESSING OPTIONS PRESENTED DURING THE SUMMER SCHOOL. NONETHELESS…

FLUXNET Synthesis Workshop February 2007 – La Thuille - Italy Different networks involved: CarboeuropeIP, Ameriflux, Fluxnet-Canada, LBA, CarboAfrica, OzFlux, TCOS Siberia, ChinaFlux, KoFlux, AsiaFlux, NECC, US-China Carbon Consortium Sponsors (not all confirmed): GTOS ILEAPS NASA DAO Global Carbon Project CarboeuropeIP Organizing Commitee: D. Baldocchi T. Boden P. Ciais B. Cook B. Law D. Papale M. Reichstein S. Running R. Valentini

FLUXNET Synthesis Workshop 2007 The workshop has been designed as a real hands-on data synthesis workshop It will be restrict to people coming from the different regional networks Particularly scientists most interested in synthesis activities should participate This does not preclude valuable post-workshop intellectual input from non-participating scientists for the scientific papers and co-authorship The harmonized global dataset will be available to all the community for other synthesis works particularly for the many topics that will be not discussed at the WS Dataset will be pre-processed in an harmonized way before the WS February 2007 – La Thuille - Italy

FLUXNET Synthesis Workshop February 2007 – La Thuille - Italy Carboeurope-IP Ecosystem Component will participate to the synthesis All the datasets submitted to the CarboeuropeIP database will be included (only if with good quality) and the PIs will be part of the activity. If you don’t want that your data will be used in this activity please send us a note. Carboeurope will have 7-8 participants to the WS and we have to provide a list of about names with topic of interest to the Organising Committee before November 30th, with if possible a good balance between experienced and young PostDoc-level scientists Based on the suggestions of all networks and in consultation with networks, the Organising Commitee will propose a final participant list before December 10th (in order to avoid gaps or overloaded topics).

FLUXNET Synthesis Workshop February 2007 – La Thuille - Italy First draft of the list: Proposed namesArea of interest M. Aubinet / C. Bernhofer / A. KnohlEnergy partitioning I. Janssen / S. Luysaaert / G. MatteucciGPP / NPP N. Buchmann / A. Granier / D. YakirEcophysiology data driven J. Grace / D. Lousteau / D. SchulzeForest age/management effect JF. Soussana / P. Smith / M. WattenbachGrassland and croplands E. Falge / J. TenhunenEcophysiology model based F. Miglietta / S. Rambal / G. SuefertMediterranean synthesis T. Laurila, A. Lindroth, T. Vesala…Boreal ecosystems synth. - NECC