1. Data Access - a common situation Need fish data! Hmm maybe Don can help… *RING RING* Hi Don, I need some upper Waimak fish observations for my model. Any ideas? Don Hmm, Ive got one dataset. Ill send it through… 10 minutes… To: Joe 01/02/09, 3.2, 3, 1 01/02/11, 3.1, 3, 1 10 minutes… *RING RING* Ok. Got the data. Where is the site located? Oh, its at laughing jack bridge. Coordinates? Ummm. (papers shuffle) 147.123 -41.588 What reference system?? I think its GDA94 Ok. What method is used? Umm, I think observations are done by Fyke net.. Oh…how does this compare to electric fishing? Umm...... DON? Joe Fish *CLICK* Modified From Peter Taylor (CSIRO)

2. The Interoperability Problem We want all information to be accessible. What does that mean? User can discover information (space, time, topic) enabled through well-defined interfaces. User can understand and assess information (relevance) for example enabled through well-defined vocabularies. User can access information through well-defined protocols. Notes:Users can be many men and robots! Information can come from many data sources

3. The Interoperability Problem Only information that is well managed is discoverable, understandable and accessible. Information management starts and information capture / generation (e.g. in the field). Federated information management and access is enabled through standards. We want all information to be accessible

4. What do we need to be able to federate biodata? Jochens thoughts

5. Towards a New Zealand interoperable biodata infrastructure - strawperson - Data Management Consistent Metadata Consistent Taxa descriptors Consistent reference information (geospatial, temporal) Consistent description of data (parameter, method, etc.) Delivery Agreed open standard web services

6. Information Management made consistent through a services stack - biodata - Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKESPI) DS2 (LAKESPI) DS3... DS3... Web Delivery Systems Dataset 1... Dataset 1... Metadata Services Metadata Services Taxonomic Information Services Observation & Measurement Names Services Procedures & Methods Information Services Consistency through Policy, Roles and Tools Consistency through Policy, Roles and Tools Standard Web Delivery Services Standard Web Delivery Services

7. How to manage data at NIWA? (excerpt) Describe metadata for your dataset according to the NIWA Metadata Management System. Describe observed variables / parameters / units according to the NIWA Measurement and Observation variables vocabulary. If you use a standard method (hint: many people / groups are using the method), ensure it is catalogued in the NIWA Procedures and Methods Register. Ensure the relevant taxa information is catalogued consistent with the NIWA Taxonomic Information Services. WHY? This will ensure the data is archived for the future, and discoverable, accessible and understandable by you your colleagues, and can be delivered through standard web interfaces.

8. Information Management made consistent through a services stack - biodata - Metadata Services Metadata Services Taxonomic Information Services Observation & Measurement Names Services Procedures & Methods Information Services Consistency through Policy, Roles and Tools Consistency through Policy, Roles and Tools Standard Web Delivery Services Standard Web Delivery Services Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKESPI) DS2 (LAKESPI) DS3... DS3... Web Delivery Dataset 1... Dataset 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1...

9. NIWA EI Delivery Vision in New Zealand: Interoperability through open standards QA/Management CADDIS QA/Management Excel QA/Management FFDB Data Collection&QA Standards Data Archiving Standards Observations / Surveys / Data collection (to standards!) Data Archive NIWA (NEMO) Data Archive Auckland Council Data Archive Wellington Council Services Stack (Set of Standard Web Services) Data User NIWA Data User Council Data User Joe Public App/Portal ei.niwa.co.nz App/Portal NIWA QGIS App/Portal LAWA Multiple Uses Interoperability Data Delivery Standards Science User (R) App/Portal ALA App/Portal LakeSPI

10. Mission of TFBIS BSS Project Dataset a Dataset e Dataset c Dataset d Dataset b Data Sources different formats, different owners, different organisations user a user b user c user d user e ? Data compilation ad-hoc, manual, repeated, costly!! BSS standard formats and vocabularies, standard metadata, standard web service delivery Dataset a Dataset e Dataset c Dataset d Dataset b use case a use case b use case c TFBIS BSS project TFBIS BSS project Data Access: Portals, Web services, queries

11. BSS project- Milestones Clearly identify use cases (for occupancy data) and barriers and document this. Define required solutions and specify BSS infrastructure. Implement infrastructure for a limited set of organisations and data sources. Role-out across New Zealand, supported through RC, NEMS, MfE, Stats NZ, DOC, Councils, etc.

12. Data Access - a future situation Need fish data! To: Joe 01/02/09, 3.2, 3, 1 01/02/16, 3.1, 3, 1 Joe Fish To: Joe 01/02/09, 3.2, 3, 1 01/02/15, 3.1, 3, 1 To: Joe 01/02/09, 3.2, 3, 1 01/02/14, 3.1, 3, 1 To: Joe 01/02/09, 3.2, 3, 1 01/02/12, 3.1, 3, 1 01/02/09, 3.2, 3, 1 01/02/11, 3.1, 3, 1 01/02/14, 3.1, 3, 1 01/03/14, 3.1, 3, 1 01/10/15, 3.1, 3, 1 01/02/09, 3.2, 3, 1 01/02/11, 3.1, 3, 1 01/02/14, 3.1, 3, 1 01/03/14, 3.1, 3, 1 01/10/15, 3.1, 3, 1 Magic BSS Button Have fish data!

14. Current world of biodata sharing in NZ Data sitting in institutional or even private archives. Data (existence) is generally not published. Data is not discoverable. Data is not reconcilable. Lots of effort ($$) in pulling data together Data source a

15. Operations of the biodata services stack Standards Guidelines National tools (e.g. vocabulary services) BSS

16. A Common Biodata Infrastructure CBI Synonyms: Biodata Services Stack …

17. Federated bio observations data system What do we need? Use of (in all linked systems): Consistent geospatial / temporal context Consistent meta data (ANZLIC, ISO) Consistent taxa data (NZOR) _____________________________ These are harder… Consistent environmental feature vocabulary? (e.g. river names, lake names, …) Consistent (institutional?) methods vocabulary (use (national) standards as reference?) Consistent observations types vocabulary? (units vocabulary, parameter vocabulary)

18. A Common Biodata Infrastructure CBI Use-Cases / Why do we need it? Users: DOC, MfE, MPI, Primary Sector, Stats NZ, Regional Councils, CRIs, NGOs Uses: States/Trends (SoE), Tier 1 statistics, Research, ??? Data Providers: CRIs, DOC, Regional Councils, Te Papa, NGOs, Citizens Information Feedback (improved models, etc) Uses determine specification Can include: Reporting Tools Products / Analytical Tools Standards Data Management Tools Monitoring Tools

19. A Common Biodata Infrastructure CBI Information Types to be managed We are dealing with Sampling data – need vocabularies to ensure consistency Sampling data types: Survey Data (Systematic, well-defined scope) geo-chrono-taxa mapping Ad-hoc data: Collections, Citizen Science Vocabularies, e.g. Taxonomies (e.g. NZOR, soil classification) Includes metadata about entities in vocabularies (e.g. origin / occurrence) Descriptive Information about entities within a vocabulary Need to ensure contextual information can be integrated! E.g. climate, land cover etc.

20. Use case: Regional Councils - HBRC Tabulation of connected environmental parameters derived through interpolation/aggregation (rules required!) x/y, date/time, parameter1, parameter 2, … Tabulations of parameters x/y, date/time, parameter 1 Tabulations of parameters x/y, date/time, parameter 2 Survey 1 Survey 2 Metadata Standards Observation and Measurement Nomenclature Taxonomic vocabulary Standardized datasets Standardized formats / templates Authoritative Data Archive 1 Authoritative Data Archive 2 Data Archiving Data Collection Data Homogenization Data QA Data Delivery Data Connection / Amalgamation Survey x..