1. Pierre Jaccard1 MyOcean Quality Control for Ferryboxes 10.06.2013 - MyOcean Tutorial, NERSC, Bergen

2. Quality Controlled Data How good are they? Is a good data good enough? Definitions 10.06.2013 - MyOcean Tutorial, NERSC, Bergen Pierre Jaccard2

3. 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 3 MyOcean In-Situ Quality Control Real Time Quality Control Procedures – Based on ARGO QC procedures Argo, 2009: Argo quality control management, Version 2.4, Argo data management – T and S QC manual – BioGeoChem QC manual Dealyed Mode Quality Control Procedures – T and S QC manual – BioGeoChem QC manual – Interaction with SDN2 Parameters – Temperature, Salinity – Chla Fluorescence, Dissolved Oxygen

4. Infrastructure Types Vertical Profiles – ARGO, XBT, CTD – Gliders, AUV Time Series – Moorings – Ferrybox 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 4

5. netCDF File Format Variables TIME, TIME_QC LONGITUDE, LATITUDE, POSITION_QC DEPH, DEPH_QC TEMP, TEMP_QC PSAL, PSAL_QC FLU2, FLU2_QC DOXY, DOXY_QC 10.06.2013 - MyOcean Tutorial, NERSC, Bergen Pierre Jaccard5

6. Variable Attributes Example for variable TEMP: long_name = sea water temperature in-situ ITS-90 scale standard_name = sea_water_temperature units = degrees_Celsius valid_min = -2.0 valid_max = 40.0 comment = QC_indicator = QC_procedure = 7 sensor_depth = ancillary_variables = TEMP_QC uncertainty = 0 accuracy = 0 precision = 0 resolution = 0 cell_methods = TIME:point DEPTH:point LATITUDE:point LONGITUDE:point DM_indicator = R ferrybox_sensor = 10.06.2013 - MyOcean Tutorial, NERSC, Bergen Pierre Jaccard6

7. QC Variables Attributes 10.06.2013 - MyOcean Tutorial, NERSC, Bergen Pierre Jaccard7 long_name = quality flag conventions = OceanSITES reference table 2 valid_min = numpy.byte: 0 valid_max = numpy.byte: 9 flag_values = func:MyO.Export.Formats.MyOcean.flags_values flag_meanings = no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used nominal_value interpolated_value missing_value

8. Flags 10.06.2013 - MyOcean Tutorial, NERSC, Bergen Pierre Jaccard8 0No QC was performedShould not be used without control 1Good dataCan be used safely 2Probably good dataCan be used safely for some applications 3Bad data, but potentially correctableShould not be used, but maight be improved later 4Bad dataShould not be used 5Value changed 6Below detection limit 7In excess of quoted value 8Interpolated value 9Missing value AIncomplete information

9. ARGO Specific Tests Platform identification (GTS) Impossible speed test (GTS) – Check drift speed of floats Digit rollover test – When number of digits is too small Grey list (ARGO) – Stop assimiliation and dissemination of data from sensors not working properly Gross salinity or temperature drift (ARGO) – Detect significant sensor drift Deepest pressure test (ARGO) – Some kind of pressure range test Instrument range test (Gliders, AUV) – Detect if instrument is within oceanographic limits 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 9

10. Vertical Profiles Tests Frozen profile test – Detect instruments that reproduce the same profile (with too little variations) Density inversion Bottom spike test (XBT) – Detect sudden temperature change Pressure increasing test 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 10

11. Common Tests Impossible date test Impossible location test Position on land test Global range test Regional range test Spike test Gradient test Stuck value test Rate of change in time test (Moorings) Calibration status check – Move data from good to probably good after a certain period of time – Delayed mode Parameter relationship test 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 11

12. Ferrybox Specific Tests Speed range test (Ferrybox) Pump test (Ferrybox) Pump history test (Ferrybox) Instrument comparison test – Detect if 2 measurements of same parameter are close Subsequent trip test – Detects and corrects sensor cleaning events (biofouling) – Delayed mode 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 12

13. Location Tests 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 13 [TIME] [[FROZEN]] consecutive measurements at identical time [[VALUE]] 1950-01-01 00:00:00 < t < now [[LENGTH]] require a minimum number of consecutive measurements minimum = 900s [SPEED] [[RANGE]] min = 0.5 max = 15.0 [[FROZEN]] Consecutive measurements with identical speed [[HISTORY]] Require sustained speed for a minimum periode of time duration = 300s [POSITION] [[FROZEN]] consecutive measurements at identical position [[RANGE]] geographic placement on earth

14. Flow Tests 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 14 [PUMP] [[VALUE]] System must be pumping [[HISTORY]] Require flow through system for a minimal period of time Duration = 600s [OBSTRUCTION] [[VALUE]] Obstruction signal must be inactive [[HISTORY]] Require inactive signal for a minimum period of time

15. Parameter Tests 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 15 [PARAMETER] [[GLOBAL_RANGE]] [[REGIONAL_RANGE]] Special range tests by geographical range and time range in year [[GRADIENT]] ARGO gradient test on T&S Modified ARGO Test on BGC Work in progress to support all parameters and platforms [[SPIKE]] ARGO gradient test on T&S Modified ARGO Test on BGC Work in progress to support all parameters and platforms

16. Regional Tests 23.04.2013 NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013 16