Pierre Jaccard1 MyOcean Quality Control for Ferryboxes MyOcean Tutorial, NERSC, Bergen

Quality Controlled Data How good are they? Is a good data good enough? Definitions MyOcean Tutorial, NERSC, Bergen Pierre Jaccard2

NIVA - Jerico/Ferrybox Worlshop Helsinki April MyOcean In-Situ Quality Control Real Time Quality Control Procedures – Based on ARGO QC procedures Argo, 2009: Argo quality control management, Version 2.4, Argo data management – T and S QC manual – BioGeoChem QC manual Dealyed Mode Quality Control Procedures – T and S QC manual – BioGeoChem QC manual – Interaction with SDN2 Parameters – Temperature, Salinity – Chla Fluorescence, Dissolved Oxygen

Infrastructure Types Vertical Profiles – ARGO, XBT, CTD – Gliders, AUV Time Series – Moorings – Ferrybox NIVA - Jerico/Ferrybox Worlshop Helsinki April

netCDF File Format Variables TIME, TIME_QC LONGITUDE, LATITUDE, POSITION_QC DEPH, DEPH_QC TEMP, TEMP_QC PSAL, PSAL_QC FLU2, FLU2_QC DOXY, DOXY_QC MyOcean Tutorial, NERSC, Bergen Pierre Jaccard5

Variable Attributes Example for variable TEMP: long_name = sea water temperature in-situ ITS-90 scale standard_name = sea_water_temperature units = degrees_Celsius valid_min = -2.0 valid_max = 40.0 comment = QC_indicator = QC_procedure = 7 sensor_depth = ancillary_variables = TEMP_QC uncertainty = 0 accuracy = 0 precision = 0 resolution = 0 cell_methods = TIME:point DEPTH:point LATITUDE:point LONGITUDE:point DM_indicator = R ferrybox_sensor = MyOcean Tutorial, NERSC, Bergen Pierre Jaccard6

QC Variables Attributes MyOcean Tutorial, NERSC, Bergen Pierre Jaccard7 long_name = quality flag conventions = OceanSITES reference table 2 valid_min = numpy.byte: 0 valid_max = numpy.byte: 9 flag_values = func:MyO.Export.Formats.MyOcean.flags_values flag_meanings = no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used nominal_value interpolated_value missing_value

Flags MyOcean Tutorial, NERSC, Bergen Pierre Jaccard8 0No QC was performedShould not be used without control 1Good dataCan be used safely 2Probably good dataCan be used safely for some applications 3Bad data, but potentially correctableShould not be used, but maight be improved later 4Bad dataShould not be used 5Value changed 6Below detection limit 7In excess of quoted value 8Interpolated value 9Missing value AIncomplete information

ARGO Specific Tests Platform identification (GTS) Impossible speed test (GTS) – Check drift speed of floats Digit rollover test – When number of digits is too small Grey list (ARGO) – Stop assimiliation and dissemination of data from sensors not working properly Gross salinity or temperature drift (ARGO) – Detect significant sensor drift Deepest pressure test (ARGO) – Some kind of pressure range test Instrument range test (Gliders, AUV) – Detect if instrument is within oceanographic limits NIVA - Jerico/Ferrybox Worlshop Helsinki April

Vertical Profiles Tests Frozen profile test – Detect instruments that reproduce the same profile (with too little variations) Density inversion Bottom spike test (XBT) – Detect sudden temperature change Pressure increasing test NIVA - Jerico/Ferrybox Worlshop Helsinki April

Common Tests Impossible date test Impossible location test Position on land test Global range test Regional range test Spike test Gradient test Stuck value test Rate of change in time test (Moorings) Calibration status check – Move data from good to probably good after a certain period of time – Delayed mode Parameter relationship test NIVA - Jerico/Ferrybox Worlshop Helsinki April

Ferrybox Specific Tests Speed range test (Ferrybox) Pump test (Ferrybox) Pump history test (Ferrybox) Instrument comparison test – Detect if 2 measurements of same parameter are close Subsequent trip test – Detects and corrects sensor cleaning events (biofouling) – Delayed mode NIVA - Jerico/Ferrybox Worlshop Helsinki April

Location Tests NIVA - Jerico/Ferrybox Worlshop Helsinki April [TIME] [[FROZEN]] consecutive measurements at identical time [[VALUE]] :00:00 < t < now [[LENGTH]] require a minimum number of consecutive measurements minimum = 900s [SPEED] [[RANGE]] min = 0.5 max = 15.0 [[FROZEN]] Consecutive measurements with identical speed [[HISTORY]] Require sustained speed for a minimum periode of time duration = 300s [POSITION] [[FROZEN]] consecutive measurements at identical position [[RANGE]] geographic placement on earth

Flow Tests NIVA - Jerico/Ferrybox Worlshop Helsinki April [PUMP] [[VALUE]] System must be pumping [[HISTORY]] Require flow through system for a minimal period of time Duration = 600s [OBSTRUCTION] [[VALUE]] Obstruction signal must be inactive [[HISTORY]] Require inactive signal for a minimum period of time

Parameter Tests NIVA - Jerico/Ferrybox Worlshop Helsinki April [PARAMETER] [[GLOBAL_RANGE]] [[REGIONAL_RANGE]] Special range tests by geographical range and time range in year [[GRADIENT]] ARGO gradient test on T&S Modified ARGO Test on BGC Work in progress to support all parameters and platforms [[SPIKE]] ARGO gradient test on T&S Modified ARGO Test on BGC Work in progress to support all parameters and platforms

Regional Tests NIVA - Jerico/Ferrybox Worlshop Helsinki April