1. Improving Access to Biological Observations data Dr. Hassan Moustahfid NOAA US IOOS Program Office Silver Spring, MD NMFS FMAC F2F Brief 08-16-2013

2. Outline IOOS BDP implementation IOOC BIO TT Animal Telemetry Data Development Acoustics Obs Active and Passive 2

3. 1.Temperature 2.Salinity 3.Water level 4.Currents 5.Surface Waves 6.Surface Winds 7.Ocean color 8.Dissolved oxygen 9.pH 10.pCO 2 11.Heat flux 12.Bottom character 13.Pathogens IOOS ® Core Variables list could be expanded based on identified needs... 14.Bathymetry 15.Ice distribution 16.Contaminants 17.Stream flow 18.Dissolved nutrients 19.Optical properties 20.Total suspended matter 21.Colored dissolved organic matter 22.Fish species 23.Fish abundance 24.Zooplankton species 25.Phytoplankton species 26.Zooplankton abundance 3

4. 4 Interoperability of Original Data: IOOS Terminology Biological Data Contents Observation (e.g. Point Data) Species Presence - Absence and Species Abundance Observation (e.g. Point Data) Species Presence - Absence and Species Abundance Record Identity / Citation Spatial-Temporal- Reference Taxon / Identification Observation Details Sampling Details Record Identity / Citation Spatial-Temporal- Reference Taxon / Identification Observation Details Sampling Details Details Record Semantics IOOS Biological Data Terminology Data Mapping Dublin Core IOOS Terminology Original Data “Mapping Document” Align originator definitions with IOOS terms “Mapping Document” Align originator definitions with IOOS terms “Fallout” and Resolu- tion Web Service Darwin Core IOOS- specific terms Metadata Development CF Courtesy of Philip Goldstein (OBIS-USA) Metadata Standards, Content and Access Metadata Access FGDC Document of Record FGDC Document of Record Metadata Contents FGDC is the Primary Format ISO and ncISO Supported Multiple Interface Methods FGDC is the Primary Format ISO and ncISO Supported Multiple Interface Methods Identification / Attribution Discovery Understanding Referral History / QA (Details of Mapping to Terminology) Identification / Attribution Discovery Understanding Referral History / QA (Details of Mapping to Terminology) Content Standard Purpose Native FGDC xml via http:// Converted ISO xml via http:// ncISO on ERDDAP FGDC & ISO; html, xml, text Tools transform FGDC for other formats and methods of access; tools by project team and external. Credit to OBIS-USA, PIFSC, NPS, PMNM and PacIOOS teams Metadata: information used to document and interpret data

5. IOOS BDP data conventions and services Metadata conventions and exchange standards -Data Model XML -http://www.ioos.gov/schema/ioosbiology/1_0/ioos_biological_terminology201206 08v1point0.xsdhttp://www.ioos.gov/schema/ioosbiology/1_0/ioos_biological_terminology201206 08v1point0.xsd -Html version -http://www.ioos.gov/schema/ioosbiology/1_0/ioos_biological_terminology201206 08v1point0.xmlhttp://www.ioos.gov/schema/ioosbiology/1_0/ioos_biological_terminology201206 08v1point0.xml -Data Access services Metadata http://oos.soest.hawaii.edu/cgi- bin/get_metadata.pl?id=PACN_FISH_TRANSECT&format=fgdc Or http://oos.soest.hawaii.edu/erddap/info/index.html?page=1&itemsPerPage=1000 5 US IOOS BDP ATN TOPP POST ACOUSTIC CHALLENGES

6. Two parallel activities: -Implementation in IOOS Gulf of Mexico and exposing SEFSC and FWC FIM Fisheries data (Different sampling gears, Trawl and >1200 species) GCOOS ERDDAP: http://barataria.tamu.edu/erddap/info/index.html?page=1&itemsPerPage= 1000 http://barataria.tamu.edu/erddap/info/index.html?page=1&itemsPerPage= 1000 -Implementation in IOOS Southeast and exposing MARMAP data – SCDNR and SEFSC SECOORA ERDDAP: http://129.252.139.124/erddap/info/index.html?page=1&itemsPerPage=1 000, http://129.252.139.124/erddap/info/index.html?page=1&itemsPerPage=1 000 -NCDDC team is working closely with us to develop FGDC and ISO metadata. 6 FY12-FY13 Implementation in IOOS regions

7. IOOS SYSTEM ARCHITECTURE FOR BIOLOGICAL DATA

8. SERVICE REGISTRY Metadata Harvester DATA CATALOG DATA VIEWER US IOOS PORTAL Backend Data storage IOOS Regional Data Nodes Bio. Core Var. Data (ERDDAP or other accepted IOOS Service) Backend Data storage Federal Agencies (NOAA NMFS, NOS, BOEM, NAVY..etc) Bio. Core Var. Data ERDDAP or other Accepted IOOS service) DATA.GOV OBIS Others data aggregators DATA USERS NODC Enrollment Formatting to IOOS DMAC Standards

9. 9 Enhancing NMFS data standardization and integration NMFS Data Catalog Metadata Harvester Data Viewer Based on Google Maps etc. RDMS In each NMFS SC or RO DMS Web services Data Encoding Or Configuring Following IOOS DMAC standards Data.Gov IOOS Catalog

10. Animal Telemetry Observations Data Content and Access Standards

11. AAT Observations System Design – Service Connections. Access to all data via ERDDAP: RDBMS > netCDF files (CDM) > ERDDAP.

12. Acoustic Metadata Convention for Archiving

13. Why? – Providing a central data depository so NOAA/NMFS scientists can safeguard and share their research data. How? – Work with data holders (e.g., NMFS Fisheries Science Centers) and NGDC Where? – NGDC database archiving system

14. Type of NMFS acoustic data Passive acoustics Active fisheries acoustic Multibeam, sidescan etc.

15. Working Plan for Acoustic Data Archiving: The First 3 Months Contact NGDC to discuss the concept of metadata standards for passive acoustic data archiving Work with NMFS Science Centers to finalize archiving plan that under NGDC data standards Pilot project to implement archiving of subset of acoustic data (if resources allow)

16. Which data model fit this type of data? 16 NetCDF (Network Common Data Form) may be a better format for storage of final acoustic output data. Sv ( Acoustic Volume Backscatter ) final data file is well suited to array-oriented acoustic data. Further, metadata contained in the NetCDF files can be automatically harvested to facilitate ready discovery of the data.

17. 17 Example. Data variable for volume backscatter and ancillary data quality variable Sv Size: 120x2154 Dimensions: DEPTH,TIME Datatype: double Attributes: name = 'Sv' long_name = 'mean_volume_backscatter_coefficient' units = 'm-1' _FillValue = 1e+04 valid_min = 0 valid_max = 1 ancillary_variables = 'Sv_quality_control' Sv_quality_control Size: 120x2154 Dimensions: DEPTH,TIME Datatype: int8 Attributes: long_name = 'quality flag for mean_volume_backscatter_coefficient' quality_control_set = 1 quality_control_conventions = 'IOOS standard flags' _FillValue = 99 valid_min = 0 valid_max = 9 flag_values = [0 1 2 3 4 5 6 7 8 9] flag_meanings = 'No_QC_performed, Good_data, Probably_good_data, Bad_data_that_are_potentially_correctable, Bad_data Value_changed, Not_used, Not_used, Not_used, Missing_value

18. 18 Metadata Convention for Passive Acoustic Recordings (PAM data)

19. 19 Metadata Convention for Active Acoustic, Fisheries Ecosounder, Sonar

20. A Clear View of Tomorrow IOOS data is helping the west coast shellfish industry mitigate the effects of ocean acidification Thank You Hassan Moustahfid, PhD. NOAA, US IOOS www.noaa.ioos.gov