1. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 1 Introduction to Exchange Formats First Data Management Training Workshop 12-17 February 2007, Oostende, Belgium Sissy IONA

2. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 2 Exchange Data Formats in SeaDataNet  CF-compliant NetCDF binary format (for gridded fields and multi- dimensional data types)  ODV ASCII spreadsheet format (for profiles, time series)  MEDATLAS ASCII format (optional)

3. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 3 NetCDF data format NetCDF (network Common Data Form) is an interface to create, access, and share array-oriented data in a form that is self-describing and portable. "Self- describing" means that a dataset includes information defining the data it contains. "Portable" means that the data in a dataset is represented in a form that can be accessed by computers with different ways of storing integers, characters, and floating-point numbers. Using the netCDF interface for creating new datasets makes the data portable. Using the netCDF interface in software for data access, management, analysis, and display can make the software more generally useful. The netCDF software includes C, Fortran, C++, Java and other interfaces for accessing netCDF data. These libraries are available for many common computing platforms.” (NetCDF Users Guide, V3.6.1, 2006, http://www.unidata.ucar.edu/software/netcdf)http://www.unidata.ucar.edu/software/netcdf Principally, designed for gridded data but extended to other observational data. NetCDF software was developed at the Unidata Program Center in Boulder, Colorado. It is freeley available at the above UCAR’s website.

4. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 4 CF metadata Convention The Climate and Forecast (CF) Conventions is a standard designed to promote the processing and sharing of NetCDF files. The conventions locate data in space-time and as a function of other independent variables, facilitate processing and graphics. Identify data sufficiently to enable users of data from different sources to decide what is comparable and to distinguish variables in archives. The CF conventions generalize and extend the COARDS conventions. The CF conventions can be contained in other formats like XML. The CF conventions includes features that describe: Grid-cell boundaries Horizontal grids other than latitude-longitude Recording common statistical operations Standardised identification of physical quantities Non-spatiotemporal axes Climatological statistics Data compression Full documentation available on http://www.cgd.ucar.edu/cms/eaton/cf-metadata/http://www.cgd.ucar.edu/cms/eaton/cf-metadata/

5. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 5 NetCDF data file structure and main CF features A NetCDF data file has four basic components: DimensionsDimensions: gives the dimension information for the variables. Dimensions can be spatial, temporal or any other quantity. The dimensions can contain regularly or irregularly space steps and there is no limit in the number of dimensions. The ‘UNLIMITED’ dimension occurs when the dimension is ‘time’ and data are still being taken. VariablesVariables: contain information on the actual variables as well as on each dimension. Coordinates also contained in variables. The variables should be named using the CF Standard Names, otherwise ‘long_name’ attribute should be used to describe the variable. CF requires all variables to have units unless they contain dimensionless numbers. Global attributesGlobal attributes: must include as much information as possible. CF conventions provide some basic “discovery” metadata in global attributes: title : description of what is in the dataset Institution : that produces the data source : the method of production of the original data - model, observational history : modifications to the original data references : publications or web references Comment : any other information DataData: stores the data. CF conventions suggest the ‘_FillValue’ attribute for missing data and defined as the same type of the variable that replaces.

6. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 6 NetCDF data file structure (http://badc.nerc.ac.uk/help/formats/netcdf/index_cf.html)

7. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 7 Example of gridded data PCMDI Sea Surface temperature data set in NetCDF format following the CF conventions: ------------------------------------------------------------------------------------------------------------------------------------------- netcdf tos_O1_2001-2002 { filename must have “.nc” extension dimensions: lon = 180 ; lat = 170 ; time = UNLIMITED ; // (24 currently) bnds = 2 ; number of vertices variables: double lon(lon); coordinate variable of 1 dimension lon:standard_name = "longitude”; legal value from CF Standard_Name table lon:long_name = "longitude" ; lon:units = "degrees_east”; compliant with CF/COARDS/Udunits lon:axis = "X”; legal value of axis are X,Y,Z and T lon:bounds = "lon_bnds“; represent the cell lon:original_units = "degrees_east" ; double lon_bnds(lon, bnds); define the extent of the cell and the grid point location double lat(lat) ; lat:standard_name = "latitude" ; lat:long_name = "latitude" ; lat:units = "degrees_north" ; lat:axis = "Y" ; lat:bounds = "lat_bnds" ; lat:original_units = "degrees_north" ; double lat_bnds(lat, bnds) ;

8. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 8 double time(time) ; time:standard_name = "time" ; time:long_name = "time" ; time:units = "days since 2001-1-1”; the time units must contain a reference time time:axis = "T" ; time:calendar = "360_day“; defines the permitted values of year,month,day time:bounds = "time_bnds" ; time:original_units = "seconds since 2001-1-1" ; double time_bnds(time, bnds) ; float tos(time, lat, lon) ; tos:standard_name = "sea_surface_temperature" ; tos:long_name = "Sea Surface Temperature" ; tos:units = "K" ; tos:cell_methods = "time: mean (interval: 30 minutes)" ; defines the variation within the cells tos:_FillValue = 1.e+20f ; tos:missing_value = 1.e+20f ; tos:original_name = "sosstsst" ; tos:original_units = "degC" ; tos:history = " At 16:37:23 on 01/11/2005: CMOR altered the data in the following ways: added 2.73150E+02 to yield output units; Cyclical dimension was output starting at a different lon;" ; Example of gridded data

9. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 9 // global attributes: :title = "IPSL model output prepared for IPCC Fourth Assessment SRES A2 experiment" ; :institution = "IPSL (Institut Pierre Simon Laplace, Paris, France)" ; :source = "IPSL-CM4_v1 (2003) : atmosphere : LMDZ (IPSL-CM4_IPCC, 96x71x19) ; ocean ORCA2 (ipsl_cm4_v1_8, 2x2L31); sea ice LIM (ipsl_cm4_v" ; :contact = "Sebastien Denvil, sebastien.denvil@ipsl.jussieu.fr" ; :project_id = "IPCC Fourth Assessment" ; :table_id = "Table O1 (13 November 2004)" ; :experiment_id = "SRES A2 experiment" ; :realization = 1 ; :cmor_version = 0.96f ; :Conventions = "CF-1.0" ; Identification of convention :history = "YYYY/MM/JJ: data generated; YYYY/MM/JJ+1 data transformed At 16:37:23 on 01/11/2005, CMOR rewrote data to comply with CF standards and IPCC Fourth Assessment requirements" ; :references = "Dufresne et al, Journal of Climate, 2015, vol XX, p 136" ; :comment = "Test drive" ; data: lon = 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 69, 71, 73,........................................................................................................................ lon_bnds = -0, 2, 2, 4, 4, 6,........................................................................................................................ Example of gridded data

10. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 10 Example of multi-dimensional data OceanSITES ADCP time series in NetCDF format compliant with the CF/COARDS standard: ------------------------------------------------------------------------------------------------------------------------------------------- netcdf gebco2005_leg1_150K_1_fhv1 { dimensions: N_DATE_TIME = 390 ; N_LEVEL = 40 ; DATE_TIME = 14 ; CONST1 = 1 ; CONST2 = 2 ; STRING32 = 32 ; STRINGFILT = 30 ; STRINGFILTUSES = 1 ; variables: char REFERENCE_DATE_TIME(DATE_TIME) ; REFERENCE_DATE_TIME:long_name = "Date of reference for Julian days" ; REFERENCE_DATE_TIME:convention = "YYYYMMDDHHMISS" ; REFERENCE_DATE_TIME:_FillValue = " " ; double JULD(N_DATE_TIME) ; JULD:long_name = "Julian days relative to REFERENCE_DATE_TIME" ; JULD:_FillValue = -999999. ; float CAS_DATE_FLAG(N_DATE_TIME) ; CAS_DATE_FLAG:long_name = "Flag on ADCP current Juld" ; CAS_DATE_FLAG:valid_min = 0s ; CAS_DATE_FLAG:valid_max = 10s ; CAS_DATE_FLAG:_FillValue = -99999.f ; double JULD_ADCP(N_DATE_TIME) ; JULD_ADCP:long_name = "Julian days ADCP relative to REFERENCE_DATE_TIME" ; JULD_ADCP:_FillValue = -999999. ; char DATE_TIME_UTC(N_DATE_TIME, DATE_TIME) ; DATE_TIME_UTC:long_name = "ASCII gregorian date and time" ; DATE_TIME_UTC:convention = "YYYYMMDDHHMISS" ; DATE_TIME_UTC:_FillValue = " " ; float LATITUDE(N_DATE_TIME) ; LATITUDE:long_name = "Latitude of each location" ; LATITUDE:units = "degree_north" ; LATITUDE:valid_min = -90.f ; LATITUDE:valid_max = 90.f ; LATITUDE:_FillValue = -999999.f ; float LONGITUDE(N_DATE_TIME) ; LONGITUDE:long_name = "Longitude of each location" ; LONGITUDE:units = "degree_east" ; LONGITUDE:valid_min = -180.f ; LONGITUDE:valid_max = 180.f ; LONGITUDE:_FillValue = -999999.f ;

11. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 11 Example of multi-dimensional data float UVEL_SHIP(N_DATE_TIME) ; UVEL_SHIP:units = "meter per second" ; UVEL_SHIP:long_name = "Eastward ship velocity" ; UVEL_SHIP:valid_min = -20.f ; UVEL_SHIP:valid_max = 20.f ; UVEL_SHIP:_FillValue = -999999.f ; float VVEL_SHIP(N_DATE_TIME) ; VVEL_SHIP:long_name = "Northward ship velocity" ; VVEL_SHIP:units = "meter per second" ; VVEL_SHIP:valid_min = -20.f ; VVEL_SHIP:valid_max = 20.f ; VVEL_SHIP:_FillValue = -999999.f ; float DEPH(N_LEVEL) ; DEPH:long_name = "Depth of bin center" ; DEPH:units = "meter" ; DEPH:valid_min = -12000.f ; DEPH:valid_max = 0.f ; DEPH:_FillValue = -999999.f ; float TEMP_ADCP(N_DATE_TIME) ; TEMP_ADCP:long_name = "ADCP transducer temperature" ; TEMP_ADCP:units = "degree_Celsius" ; TEMP_ADCP:valid_min = -5.f ; TEMP_ADCP:valid_max = 45.f ; TEMP_ADCP:_FillValue = -999999.f ; float HDG(N_DATE_TIME) ; HDG:long_name = "Ship heading" ; HDG:units = "degree" ; HDG:valid_min = -360.f ; HDG:valid_max = 360.f ; HDG:_FillValue = -999999.f ; float PTCH(N_DATE_TIME) ; PTCH:long_name = "Ship pitch" ; PTCH:units = "degree" ; PTCH:valid_min = -360.f ; PTCH:valid_max = 360.f ; PTCH:_FillValue = -999999.f ; float ROLL(N_DATE_TIME) ; ROLL:long_name = "Ship roll" ; ROLL:units = "degree" ; ROLL:valid_min = -360.f ; ROLL:valid_max = 360.f ; ROLL:_FillValue = -999999.f ;

12. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 12 Example of multi-dimensional data float CAS_CURRENT_FLAG(N_DATE_TIME, N_LEVEL) ; CAS_CURRENT_FLAG:long_name = "Flag on ADCP current (U,V,W)" ; CAS_CURRENT_FLAG:valid_min = 0s ; CAS_CURRENT_FLAG:valid_max = 10s ; CAS_CURRENT_FLAG:_FillValue = -99999.f ; float VVEL_ADCP(N_DATE_TIME, N_LEVEL) ; VVEL_ADCP:long_name = "Northward absolute ADCP current velocity" ; VVEL_ADCP:units = "meter per second" ; VVEL_ADCP:valid_min = -20.f ; VVEL_ADCP:valid_max = 20.f ; VVEL_ADCP:_FillValue = -999999.f ; float UVEL_ADCP(N_DATE_TIME, N_LEVEL) ; UVEL_ADCP:long_name = "Eastward absolute ADCP current velocity" ; UVEL_ADCP:units = "meter per second" ; UVEL_ADCP:valid_min = -20.f ; UVEL_ADCP:valid_max = 20.f ; UVEL_ADCP:_FillValue = -999999.f ; float WVEL_ADCP(N_DATE_TIME, N_LEVEL) ; WVEL_ADCP:long_name = "Upward absolute ADCP current velocity" ; WVEL_ADCP:units = "meter per second" ; WVEL_ADCP:valid_min = -20.f ; WVEL_ADCP:valid_max = 20.f ; WVEL_ADCP:_FillValue = -999999.f ; float EVEL_ADCP(N_DATE_TIME, N_LEVEL) ; EVEL_ADCP:long_name = "Absolute ADCP current velocity error" ; EVEL_ADCP:units = "meter per second" ; EVEL_ADCP:valid_min = -20.f ; EVEL_ADCP:valid_max = 20.f ; EVEL_ADCP:_FillValue = -999999.f ; float PGOOD_ADCP_B1(N_DATE_TIME, N_LEVEL) ; PGOOD_ADCP_B1:long_name = "Percent of good data before ensemble averaging" ; PGOOD_ADCP_B1:units = "percent" ; PGOOD_ADCP_B1:comment = "Number of good data = PGOOD * NB_ENS_AVE" ; PGOOD_ADCP_B1:valid_min = 0.f ; PGOOD_ADCP_B1:valid_max = 100.f ; PGOOD_ADCP_B1:_FillValue = -999999.f ; float PGOOD_ADCP_B2(N_DATE_TIME, N_LEVEL) ; PGOOD_ADCP_B2:long_name = "Percent of good data before ensemble averaging" ; PGOOD_ADCP_B2:units = "percent" ; PGOOD_ADCP_B2:comment = "Number of good data = PGOOD * NB_ENS_AVE" ; PGOOD_ADCP_B2:valid_min = 0.f ; PGOOD_ADCP_B2:valid_max = 100.f ; PGOOD_ADCP_B2:_FillValue = -999999.f ; …………………………………………………………………………………………………………………………..

13. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 13 Example of multi-dimensional data ………………………………………………………………………………………………………………………….. Float VVEL_ADCP_CORTIDE(N_DATE_TIME, N_LEVEL) ; VVEL_ADCP_CORTIDE:units = "meter per second" ; VVEL_ADCP_CORTIDE:long_name = "Northward absolute velocity corrected for tide" ; VVEL_ADCP_CORTIDE:valid_min = -20.f ; VVEL_ADCP_CORTIDE:valid_max = 20.f ; VVEL_ADCP_CORTIDE:_FillValue = -999999.f ; char FILT_TYPE(STRINGFILT) ; FILT_TYPE:long_name = "Type of filtering" ; char FILT_FLAGS(STRINGFILTUSES) ; FILT_FLAGS:long_name = "List of flags used before filtering" ; // global attributes: :DATE_CREATION = "26-Oct-2005" ; :SOFTWARE = "CascadeVersion 5.3" ; :ADCP_CONSTRUCTOR = "RDI" ; :ADCP_TYPE = "Ocean Surveyor" ; :CONVENTIONS = "OceanSite dictionary" ; :CRUISE_NAME = "gebco2005" ; :PLATFORM_NUMBER = "FABB" ; :PLATFORM_NAME = "BEAUTEMPS-BEAUPRE" ; :DATE_UPDATE = "26-Oct-2005" ; :DATA_TYPE = "SADCP" ; :FORMAT_VERSION = "1.0" ; :NAVIGATION_REFERENCE = "GPS Navigation" ; :HEADING_REFERENCE = "GPS Heading" ; data: REFERENCE_DATE_TIME = "19500101000000" ; JULD = 20361.5601215279, 20361.5670601851, 20361.5739872684, 20361.5809317129, 20361.5878761574, 20361.5948379631, 20361.6017824076, 20361.6087268516, 20361.6156712961, 20361.6226157406, 20361.6295775464, 20361.6365219909, 20361.6434664354, 20361.6503935186, 20361.6573379631, 20361.6642997684, 20361.6712442129, 20361.6781886574, 20361.6851157406, 20361.6920601851, 20361.6990219909, 20361.7059664354, 20361.7129108794, …………………………………………………………………………………………………………………………..

14. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 14 ODV Generic Spreadsheet format ODV (Ocean Data View) is a simple, flexible, user friendly ASCII format. It consist of 3 parts: Comment lines Header line Data lines: up to 50 values The ODV QC flags are: 0=good, 1=unknown, 4=questionable, 8=bad It supports files in ASCII and binary NetCDF format: water profile data, benthic core profile data, Surface underway data, Moored instrument time series Developed by Schlitzer, R., http://odv.awi.dehttp://odv.awi.de

15. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 15 Example of a CTD vertical profile // Data from: C:/sissy/ODV/tests/New_ODV_Collection (Feb/09/2007 11:59:08) // Extracted: Feb/09/2007 12:00:17 by Iona@SISSY Cruise;Station;Type;mon/day/yr;hh:mm;Longitude [degrees_east];Latitude [degrees_north];Bot. Depth [m];Pressure [decibar];QF;Temperature [degrees Celsius];QF;Salinity [psu];QF;Oxygen [ml/l];QF 36AE_GN36199402802;00010;C;6/1/1994;06:30;22.5;36.25;2386;8;0;19.734;0;38.703;0;'9999';8 36AE_GN36199402802;00010;C;6/1/1994;06:30;22.5;36.25;2386;9;0;19.683;0;38.702;0;'9999';8 36AE_GN36199402802;00010;C;6/1/1994;06:30;22.5;36.25;2386;10;0;19.62;0;38.713;0;'9999';8 36AE_GN36199402802;00010;C;6/1/1994;06:30;22.5;36.25;2386;11;0;19.442;0;38.717;0;'9999';8 36AE_GN36199402802;00010;C;6/1/1994;06:30;22.5;36.25;2386;12;0;19.052;0;38.728;0;'9999';8 36AE_GN36199402802;00010;C;6/1/1994;06:30;22.5;36.25;2386;13;0;18.662;0;38.739;0;'9999';8 36AE_GN36199402802;00010;C;6/1/1994;06:30;22.5;36.25;2386;14;0;18.401;0;38.808;0;'9999';8 …………………………………………… 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;1;0;20.924;0;38.556;0;'9999';8 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;2;0;20.871;0;38.547;0;'9999';8 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;3;0;20.888;0;38.536;0;'9999';8 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;4;0;20.836;0;38.541;0;'9999';8 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;5;0;20.772;0;38.541;0;'9999';8 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;6;0;20.674;0;38.546;0;'9999';8 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;7;0;20.597;0;38.539;0;'9999';8 36AE_GN36199402802;00020;C;6/1/1994;12:00;22.5;36;3100;8;0;20.527;0;38.538;0;'9999';8 ……………………………………………

16. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 16 Example of a Current Meter time series / Data from: C:/sissy/ODV/tests/New_ODV_Collection2 (Feb/09/2007 12:12:51) // Extracted: Feb/09/2007 12:14:15 by Iona@SISSY Cruise;;;Type;mon/day/yr;hh:mm;Longitude [degrees_east];Latitude [degrees_north];Bot. Depth [m];Time [years since 1900-01-01 00:00];QF;Year;QF;Month;QF;Day;QF;Time [hhmmss];QF;Current East Component [m/s];QF;Current North Component [m/s];QF;Horizontal Current Speed [m/s];QF;Direction rel. true North [degree];QF 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2228;1;1997;0;3;0;23;0;80000;0;0.009;0;-0.015;0;0.017;0;149.4;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2229;1;1997;0;3;0;23;0;83000;0;0.008;0;-0.012;0;0.014;0;146.3;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2229;1;1997;0;3;0;23;0;90000;0;0.013;0;-0.011;0;0.017;0;128.8;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.223;1;1997;0;3;0;23;0;93000;0;0.011;0;-0.009;0;0.014;0;128.8;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2231;1;1997;0;3;0;23;0;100000;0;0.011;0;-0.013;0;0.017;0;137.5;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2231;1;1997;0;3;0;23;0;103000;0;0;0;-0.014;0;0.014;0;180.6;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2232;1;1997;0;3;0;23;0;110000;0;0.005;0;-0.013;0;0.014;0;157.1;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2232;1;1997;0;3;0;23;0;113000;0;0.008;0;-0.012;0;0.014;0;146.9;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2233;1;1997;0;3;0;23;0;120000;0;0.01;0;-0.014;0;0.017;0;144.5;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2233;1;1997;0;3;0;23;0;123000;0;0.011;0;-0.008;0;0.014;0;125.6;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2234;1;1997;0;3;0;23;0;130000;0;0.007;0;-0.012;0;0.014;0;151.5;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2235;1;1997;0;3;0;23;0;133000;0;0.007;0;-0.012;0;0.014;0;148;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2235;1;1997;0;3;0;23;0;140000;0;0.013;0;-0.006;0;0.014;0;116.5;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2236;1;1997;0;3;0;23;0;143000;0;0.022;0;-0.007;0;0.023;0;107;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2236;1;1997;0;3;0;23;0;150000;0;0.027;0;-0.005;0;0.028;0;101.1;0 36AE_GN36199724603;M1N21;C;3/23/1997;08:00;24.6;40.0725;500;97.2237;1;1997;0;3;0;23;0;153000;0;0.03;0;-0.009;0;0.031;0;106;0

17. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 17 MEDATLAS format MEDATLAS format designed by the MEDATLAS and MODB consortia in conformity with the ICES/IOC GETADE recommendations in the frame of the EU MAST II programme. It is an auto-descriptive ASCII format. Main characteristics of MEDATLAS format: - facilitate the reading of the data (and not to optimize the data archiving neither to speed up the data processing). - is independent of the computer. - is flexible and accept (almost) any number of different parameters. - keeps track of the history of the data including the data collection and the processing. - allows the processing of each profile independently. Therefore the date, time and geographical co-ordinate must be reported on each profile header (and not in separate files) -real (floating) numbers must remain in the same way as they have been transmitted, not reformatted into integers. The number of decimals must implicity indicate the accuracy of the measurements

18. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 18 Data and Metadata Structure A MEDATLAS file includes: - data from the same cruise - data measured with the same instrument (CTD, Bottle, Current Meter, etc) A MEDATLAS file consists of three parts:  a cruise header based on the international ROSCOP informationcruise header  a station header including the cruise reference, the originator station reference within the cruise, date, location, list of observed parameters with unitsstation header  the data of the stationdata (example of a time series data file)time series The sequence ‘station header + data records' is repeated for each profile. The archived parameters are the observed. Derived parameters (eg. density) are not archived. Detailed description of the format and the main fields in the “Medar-Medatlas Protocol, Part I: Exchange format and quality checks for observed profiles”

19. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 19 Data and Metadata Structure MEDATLAS Vertical Profile

20. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 20 *FI35200510011 EMERLIS XXXX UNKNOWN 14/12/2005 17/12/2005 CORAL SEA 35 IRD VEGA Andres Project=PROGRAMME NATIONAL D'ECOLOGI Regional Archiving= FI Availability=P Data Type=D01 n= 3 QC=Y COMMENT COastal DYnamics Study Mooring : ------------------------------- CoDyS is a low coast mooring to measure upper layer currents and temperature in order to study the New Caledonia coastal upwelling dynamics from 14 december 2005 to 10 April 2006. The mooring instruments were : - 7 RBR temperature recorders at the depths of 30, 45, 60, 75, 90, 115 and 155 m - 3 Aanderaa RCM7 current meters at 30, 85 and 160 m. Web site : --------- http://www.ird.nc/UR65/Vega/UpwellingPage/CoDyS.html DM=HCSP HCDT EWCT NSCT controlled with no climatology *FI3520051001100300 Data Type=D01 *DATE=14122005 TIME=2200 LAT=S23 05.68 LON=E166 46.86 DEPTH=865 QC=1111 *NB PARAMETERS=08 RECORD LINES=05514 *YEAR YEAR (yyyy) def.= 9999 *MNTH MONTH (mm) def.= 99 *DAYX DAY WITHIN MONTH (dd) def.= 99 *TIME TIME WITHIN DAY (hhmmss) def.= 999999 *EWCT CURRENT EAST COMPONENT (meter/second) def.=-99.999 *NSCT CURRENT NORTH COMPONENT (meter/second) def.=-99.999 *HCSP HORIZONTAL CURRENT SPEED (meter/second) def.= 9.99 *HCDT DIRECTION REL. TRUE NORTH (degree) def.= 999 *GLOBAL PROFILE QUALITY FLAG=1 GLOBAL PARAMETERS QC FLAGS=11111111 *DC HISTORY=RCM7 AANDERAA * *DM HISTORY * *COMMENT *ADDITIONAL INFORMATION - TIME SERIES *EDATE=08042006 ETIME=1830 ELAT=S23 05.68 ELON=E166 46.86 EDEPTH=865 QC=1111 *SENSOR DEPTH=30 (metre) DISTANCE TO BOTTOM=835 (metre) QC=11 *DURATION=114 (day) QC=1 *SAMPLING RATE=1800 (second) MAGNETIC DECLINATION=13.0 (degree) * *SURFACE SAMPLES= * *YEAR MNTH DAYX TIME EWCT NSCT HCSP HCDT 2005 12 14 220000 +4.884 -4.717 6.79 134 11111111 2005 12 14 223000 +3.856 -3.723 5.36 134 11111111 2005 12 14 230000 +5.048 -3.668 6.24 126 11111111 2005 12 14 233000 +5.629 -3.797 6.79 124 11111111 Data and Metadata Structure MEDATLAS time series

21. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 21 MEDATLAS format identifiers Cruise reference (unique): FI35199745003 (String of 13 Characters, No blanks, ‘0’ instead) FI data centre code 35 GF3 country code of the data source 1997 year of the beginning of the cruise 45003 assigned to the cruise by the data centre Station reference (unique): FI3519974500300011 (String of 18 Characters, No blanks, ‘0’ instead) FI35199745003 cruise reference 0001 station name 1 cast number

22. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 22 Reference codes Reference codes are used to describe the metadata: 1.IOC/GF3 Country codes (http://www.ices.dk/ocean/codes/Country.htm) 2.ICES Ship codes (http://www.ices.dk/ocean/codes/shipcodes.htm) 3.Extended GF3 codes for parameters and units (UNESCO/IOC/IODE, 1987 : GF3 A General Formatting System for Geo-Referenced Data - Manual and Guides 17)GF3 codes 4.IOC/ROSCOP codes for data types (http://www.ices.dk/ocean/roscop/par-cod.htm)IOC/ROSCOP 5.Mediterranean and Black Seas region from International Hydrographic Bureau definitions (I.H.B.) (http://www.ices.dk/Ocean/codes/ihb.htm) 6.MEDAR/MEDATLAS Data Centres and Confidentiality Codes (MEDAR/MEDATLAS protocol).Confidentiality Codes 7.Quality flagsQuality flags

23. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 23 ROSCOP codes

24. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 24 Extended GF3 Basic list of parameters

25. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 25 Medatlas flag scale ( based on the GTSP Project definition) 2 Value Inconsistent with Statistics (out of narrow range limits) 0 Not Controlled Value 1 Correct Value 2 Value Inconsistent with Statistics (out of narrow range limits) 3 Dubious Value (spike) 4 False Value (out of broad range regional limits or corresponding to a vertical unstability,..) 5 Value Modified During QC (only for obvious location or time errors) 9 No Observed Value Medatlas Confidentiality codes P Public Domain Data L Limited Access To Data (Project Only) C Confidential Data (Submitted To Chief Scientist Permission)

26. First Data Management Training Workshop, 12-17 February, 2007, Oostende, Belgium 26 References/links -“Climate and forecast (CF) metadata convention” by J. Gregory, Univ. of Reading, Met Office Hadley Centre, UK -Coriolis data-center, Volume 2/2, V1.2, 2001 -Jonathan Gregory, 2003, “The CF metadata standard” -“Medar-Medatlas Protocol, Part I: Exchange format and quality checks for observed profiles”, V3, 2001 -NetCDF Users Guide, V3.6.1, 2006 -OceanSITES User’s Manual, V1.0, 2006 -Schlitzer, R., Ocean Data View, http://odv.awi.de, 2006http://odv.awi.de -http://badc.nerc.ac.ukhttp://badc.nerc.ac.uk - http://www.cgd.ucar.edu/cms/eaton/cf-metadata/http://www.cgd.ucar.edu/cms/eaton/cf-metadata/ -http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.htmlhttp://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html -http://www.ices.dk/oceanhttp://www.ices.dk/ocean -http://www.unidata.ucar.edu/software/netcdfhttp://www.unidata.ucar.edu/software/netcdf