1. DATA AGGREGATION AND PRODUCTS GENERATION
EMODnet Chemistry 3
DATA AGGREGATION AND PRODUCTS GENERATION
Black Sea
Luminiţa Buga
NIMRD, Romania
Kick off Meeting, May , 2017, Trieste, Italy

2. EMODnet Chemistry Summary Results – EMODnet 2
Data harvest – regional pool – Black Sea
Data aggregation & QC
Data coverage (space & time)
Products generation-DIVA Analysis
EMODnet Chemistry 3
Data and metadata to be made available - Black Sea
WP 3: Generation of data products

3. EMODnet Chemistry 2 – data harvest
Black Sea buffer data collection
(Fertilizers & Silicates)
49910 files in ODV format
22 CDI-partners; 35 data originators
Black Sea buffer data collection
(Oxygen, Chlorophyll, Acidity)
41532 files in ODV format
24 CDI-partners; 39 data originators
Black Sea buffer data collection
(Contaminants)
24343 files in ODV format
12 CDI-partners; 18 data originators

4. Black Sea buffer data collection (Fertilizers & Silicates )
Special Windows 64bit ODV
ocean_depth_profiles
ocean_pressure_profiles
time_series
Initial data aggregations of data with
extended metadata (data.csv) with ODV files 
ocean_depth_profiles
39408 stations
ocean_pressure_profiles
791 stations
time_series
7 stations
TOTAL imported stations vs received
Not all extended metadata (data.csv) were associated to the Local_CDI_ID

5. Black Sea buffer data collection (Oxygen, Chlorophyll, Acidity)
Special Windows 64bit ODV
ocean_pressure_profiles
ocean_depth_profiles
time_series
Initial data aggregations of data with
extended metadata (data.csv) with ODV files 
ocean_depth_profiles
39082 stations
ocean_pressure_profiles
654 stations
time_series
11 stations
TOTAL imported stations vs received
Not all extended metadata (data.csv) were associated to the Local_CDI_ID

6. Black Sea buffer data collection
(Contaminants)
Special Windows 64bit ODV
ocean_pressure_profiles
ocean_depth_profiles
ocean_depth_profiles (2)
time_series
Initial data aggregations of data with
extended metadata (data.csv) with ODV files 
ocean_depth_profiles
12915 stations
ocean_depth_profiles (2)
3 stations
ocean_pressure_profiles
32 stations
time_series
17 stations
TOTAL imported stations vs received
Not all extended metadata (data.csv) were associated to the Local_CDI_ID

7. Problems during Black Sea data aggregation
Errors at importing in ODV
ODV format:
Mismatches between semantic header and header
Incorrect Semantic header format
Invalid / wrong parameter code SDN:P01/SDN:P06
Empty files
Empty columns and / or extra columns
Missing quality flag column for variable
Missing Depth in Semantic header and / or in header and / or in values
Locations on land
Incorrect date format
Duplicates: Multiple LOCAL_CDI_Id in one ODV file
Mismatches between Local_CDI_Id in ODV file >> Local_CDI_Id in coupling
Originator EDMO_Code instead of Partner EDMO_Code (ODV files)

8. Aggregated_Data_ODV Collections
Errors corrected
(partner collaboration)
New import in ODV
ocean_depth_profiles
ocean_pressure_profiles
time_series
Each ODV Collection exported as SDN Aggregated ODV Collection
aggregated variables codes from P01 to P35
aggregated units
Each SDN Aggregated ODV Collection collection imported in ODV
Check again for possible not aggregated variables (P01)
ocean_pressure_profiles merged with ocean_depth_profiles
Aggregated_Data_ODV Collections

9. BS_Aggregated_Data_ODV Collection
Preliminary QC
BS_Aggregated_Data_ODV Collection
data = 0 confirmed as missing value  QV=5
data  0, real detection limit values QV=6  QV=6
data = 0  QV=6
obvious bad data (-999, 999, all profile with same value)  QV=4
impossible data (too high)  check units with data providers
all QV = 0  V=1
New check for spikes, out of range, questionable data

10. QC aggregated Black Sea data buffer
Spatial distribution
Time distribution
Vertical profiles
Phosphate
22728 CDIs
Silicate
19024 CDIs

11. QC aggregated Black Sea data buffer
Spatial distribution
Time distribution
Vertical profiles
Ammonium
6935 CDIs
Nitrate plus nitrite
9354 CDIs

12. QC aggregated Black Sea data buffer
Spatial distribution
Time distribution
Vertical profiles
Nitrate
8274 CDIs
Nitrite
11619 CDIs

13. QC aggregated Black Sea data buffer
Spatial distribution
Time distribution
Vertical profiles
Total nitrogen
3005 CDIs
Total phosphorus
3186 CDIs

14. QC aggregated Black Sea data buffer
Spatial distribution
Time distribution
Vertical profiles
Oxygen
34383 CDIs
Chlorophyll-a
1658 CDIs

15. QC aggregated Black Sea data buffer
Spatial distribution
Time distribution
Vertical profiles pH
20270 CDIs
Total number of CDIs for nutrients, Silicates, Oxygen, Chlorophyll and pH in the Black Sea.
Parameter
Phosphate
Silicate
Ammonium
Nitrate+nitrite
Nitrates
Nitrites
No CDIs
22728
19024
6935
9354
8274
11619
Parameter
Total Nitrogen
Total phosphorus
Oxygen
Chlorophyll-a pH
No CDIs
3005
3186
34383
1658
20270

16. Metadata enriched QC ODV collections sent to Central Buffer and used for EMODNET Chemistry - Dynamic Plots

17. 10-years running averages depending on data availability
Products generation-Diva Analysis
Seasons
Winter: December - February
Spring: March - May
Summer: June – August
Autumn: September - November
10-years running averages
IODE standard levels
depending on data availability
Water body phosphates:
-300 m
Water body silicate:
-200 m
Water body nitrate:
-150 m
Water body ammonium:
-100 m (BS Western slope)
Water body NO3+NO2:
Water body oxygen :
-250 m
Water body chlorophyll-a:
-20 m

18. DIVA Settings Lc: correlation: 1.5
Signal to noise ratio of the whole dataset: 0.5
ireg: mode selected for background field: mean of data
Cleaning data on mesh: cleaning data out of the mesh & outliers elimination
Parameters estimation and vertical filtering: both correlation length and signal-to-noise ratio parameters estimated using data mean distance as a minimum (for L), and both parameters vertically filtered
Analysis and reference field: analysis with data transformation: log(data)-exp(analysis)

19. 7 DIVA products in SeaDataNet NetCDF format were prepared and uploaded to the OceanBrowser Viewing Service EMODNET Chemistry Map Server (
Phosphate - Autumn
Chl_a - Summer
Oxygen - Spring
Silicate - Winter

20. Nitrate - Winter
Nitrate + Nitrite - Spring
Ammonium - Summer

21. ….. and the corresponding XML metadata on Sextant

22. QC Contaminants aggregated Black Sea
data buffer

23. Initial Contaminants dataset > 400 P01 Terms
Aggregation to P35 Vocabulary (only contaminants relevant for MSFD)

28. Stations density distribution
QC Contaminants aggregated Black Sea data buffer
Spatial distribution
Time distribution
Stations density distribution
Water
18303 CDIs
Sediment
10325 CDIs

29. Stations density distribution
QC Contaminants aggregated Black Sea data buffer
Spatial distribution
Time distribution
Stations density distribution
Biota
138 CDIs

30. Metadata enriched QC ODV collections sent to Central Buffer and used for EMODNET Chemistry - Dynamic Plots

31. Black Sea EMODnet Chemistry 3
Data and metadata to be made available for Lot 4 - Chemistry:
Measurements of concentration of chemicals in seawater, sediments and biota, particularly those relevant to the Marine Strategy Framework Directive or to global climate change
Group
Examples
Water
Sed.
Biota
Pesticidides and biocides
DDT, HCB
Heavy metals
mercury, cadmium, lead
Hydrocarbons
anthracene, fluoroanthene
Radionuclides
137Cs, 239Pu
Fertilisers
nitrogen (DIN, TN), phosphorus (DIP, TP)
Organic
total carbon (TOC)
Chlorophyll
Silicates
Partial pressures of dissolved gases
oxygen, carbon dioxide
Plastics
polyethelyne, polypropylene
Acidity (pH, pCO2, TIC, alkalinity) pH
Black Sea
marine
beach
Marine litter (2011 – 2017)
Nutrient loads (nitrogen and phosphorus ) into seas by major rivers

32. WP3: Generation of data products covers the
EMODnet Chemistry 3
WP3: Generation of data products
covers the
Task 2: Construct products from one or more data sources that provide users with information about the distribution of parameters in time and space

33. Responsibilities for regional coordinators:
EMODnet Chemistry 3
Responsibilities for regional coordinators:
To generate harmonised, aggregated and validated data collections for eutrophication and contaminants parameters
To generate updated interpolated basin DIVA maps for nutrients, chlorophyll and oxygen, including revised and latest data entries
To generate high resolution DIVA maps for nutrients near rivers mouths

34. Thank you for your attention!