Data Access - a common situation Need fish data! Hmm maybe Don can help… RING RING Hi Don, I need some upper Waimak fish observations for my model. Any.

Slides:

Advertisements

Similar presentations

National Seminar on Developing a Program for the Implementation of the 2008 SNA and Supporting Statistics in Turkey Mahmut MOL 10 September 2013 Ankara.

Advertisements

Cario, April, 18th 2005 Dr. Roland M. Wagner SDI NRW Joint Project 2004: Identification of enhanced SDI elements Dr. Roland M. Wagner

ASIAES Project Overview Satellite Image Network for Natural Hazard Management in ASEAN+3 region Pakorn Apaphant Geo-Informatics and Space Technology Development.

1 The Collaborative, Systemwide Monitoring and Evaluation Project (CSMEP) and StreamNet NW Power & Conservation Council, Sept. 20, 2006.

CPIC Training Session: Enterprise Architecture

1 ShareGeo Discovering and Sharing Geospatial Data

Metadata workshop, June The Workshop Workshop Timetable introduction to the Go-Geo! project metadata overview Go-Geo! portal hands on session.

© Statistisches Bundesamt, B 2 / Institut für Forschung und Entwicklung in der Bundesstatistik Statistisches Bundesamt ESSnet on Preparation of Standardisation.

Smarter Information Management Presenter: Dianne Macaskill Chief Executive Archives New Zealand.

Project Scope Management

Enhanced Data Description for End Users ScribeKey, LLC Brian Hebert, Solutions Architect

The Process of Data Ingestion in ÆKOS Andrew Graham and Matt Schneider TERN Ecoinformatics Data Analysts Logos used with consent. Content of this presentation.

Expert Group Meeting 2005 Centre for Spatial Data Infrastructures and Land Administration Department of Geomatics, The University of Melbourne SDI Requirements.

02-Oct-2008 European Forum for GeoStatistics 2008 in Bled Concept for an Integrated Web Solution / an Infrastructure for Geostatistics (Subproject 3)

Visions of Australia – Regional Exhibition Touring Fund Applicant organisation Exhibition title Exhibition Sample Support Material Instructions 1) Please.

Affinity Diagrams.

Connecting People With Information DoD Net-Centric Services Strategy Frank Petroski October 31, 2006.

Dr Matthew Stiff CEH Director Environmental Informatics Presentation to CRM SIG NeSC Edinburgh 12 July 2007 The Environmental Informatics Programme.

Vocabulary Services “Huuh - what is it good for…” (in WDTS anyway…) 4 th September 2009 Jonathan Yu CSIRO Land and Water.

® Initiatives in Interoperability in New Zealand Compiled for OGC Hydro DWG Arlington, March 2014 By Jochen Schmidt, NIWA, New Zealand On behalf of many.

Spatially enabling Northern Ireland Dr Suzanne McLaughlin DFP Land & Property Services GIS Ireland Conference 11 th October 2012.

Introduction to OBIS-USA Biological Data, Applications, & Relationships March 14, 2011.

NIWA National Science Centre for Environmental Information Jochen Schmidt, Chief Scientist Federated Information Infrastructure.

Nationally Significant Databases and Collections Providers’ Group Emma Kelly Environmental Information Advisor Environmental Monitoring and Reporting Team.

Design central EMODnet portal Objectives, Technical Proposal and Consultation Process.

Using SAS® Information Map Studio

Vers national spatial data infrastructure training program Value of Metadata Introduction to Metadata An overview of the value of metadata to.

Don Robertson NIWA Wellington NZ Freshwater Fish Database.

Data discovery and data processing for environmental research infrastructures Roberto Cossu ENVRI WP4 leader ESA.

Ownership and access to data deposited to a public-good archive: - Whose data is it anyway! Larry Burrows Landcare Research New Zealand Ltd.

® Hosted and Sponsored by Initiatives in Interoperability in New Zealand 80th OGC Technical Committee Austin, Texas (USA) Jochen Schmidt, NIWA, New Zealand.

A rhetorical approach to environmental information sharing Dr. Andrew Woolf, Bureau of Meteorology.

Statistics New Zealand’s End-to-End Metadata Life-Cycle ”Creating a New Business Model for a National Statistical Office if the 21 st Century” Gary Dunnet.

Joint Information Systems Committee Supporting Higher and Further Education Rachel Bruce Programme Manager, JISC Executive Collection.

1-2-3 February 2006 –Page 1 Mersea Integrated System How to improve Access/Downloading services ? How far do we go in terms of standardization ?

Digitization – Basics and Beyond workshop Interoperability of cultural and academic resources New services for digitized collections Muriel Foulonneau.

Rob Walker The INSPIRE metadata regulations and quality issues – a user view Rob Walker Association for Geographic Information, London.

Resource Description and Access (RDA) information session Deirdre Kiorgaard Australian Committee on Cataloguing Representative to the Joint Steering Committee.

The US Long Term Ecological Research (LTER) Network: Site and Network Level Information Management Kristin Vanderbilt Department of Biology University.

Archiving microdata Standards and good practices United Nations Statistics Commission New York, February 26, 2009 Olivier Dupriez World Bank, Development.

Copyright 2010, The World Bank Group. All Rights Reserved. Recommended Tabulations and Dissemination Section B.

Joseph Lukhwareni Statistics South Africa Reengineering projects focusing on metadata and the statistical cycle Statistics South Africa, South Africa 3-5.

Harmonisation of water observations data standards Pete Taylor 29 th September OGC TC – Darmstadt 2009 Water for a Healthy Country.

1 Data Management and Information Delivery The Data Management and Information Delivery (DMID) Project 10 Apr 2008 Ashwell Jenneker & Matile Malimabe.

Global Change Master Directory (GCMD) Mission “To assist the scientific community in the discovery of Earth science data, related services, and ancillary.

Managing data to maximise value Supporting flexible and efficient production of official statistics Adam Brown December 2012.

NVS New Zealand National Vegetation Survey. What is NVS? NVS (National Vegetation Survey) – New Zealand’s largest archive facility for plot-based vegetation.

Background information LakeSPI was designed in 2002 and has been available to the public via the web reporting pages since then. lakespi.niwa.co.nzakespi.niwa.co.nz.

1 Geospatial Standards for Canada Proposed blueprint for Jean Brodeur and Cindy Mitchell.

IPDA Architecture Project International Planetary Data Alliance IPDA Architecture Project Report.

Considerations for Regional Data Collection, Sharing and Exchange Bruce Schmidt StreamNet Program Manager Pacific States Marine Fisheries Commission Presentation.

CEOS Working Group on Information System and Services (WGISS) Data Access Infrastructure and Interoperability Standards Andrew Mitchell - NASA Goddard.

GCI Architecture GEOSS Information System Meeting 20 September 2013, ESA/ESRIN (Frascati, Italy) M.Albani (ESA), D.Nebert (USGS/FGDC), S.Nativi (CNR)

The new FFDB – an outlook

GBIF Implementation Plan Highlights

EOSC MODEL Pasquale Pagano CNR - ISTI

Flanders Marine Institute (VLIZ)

Towards connecting geospatial information and statistical standards in statistical production: two cases from Statistics Finland Workshop on Integrating.

Integrating Geospatial Elements into the ABS Information Model

Sub-regional workshop on integration of administrative data, big data

Prepared by: Jennifer Saleem Arrigo, Program Manager

Design central EMODnet portal Objectives and Technical description Initial draft prepared by the Flanders Marine Institute.

2. An overview of SDMX (What is SDMX? Part I)

WGISS Connected Data Assets Oct 24, 2018 Yonsook Enloe

LOD reference architecture

Descriptors of service granularity

21 November Data Science Capabilities

MSDI training courses feedback MSDIWG10 March 2019 Busan

Hydrological data - Federated infrastructure

Cultivating Semantics for Data in Agriculture and Nutrition

Presentation transcript:

Data Access - a common situation Need fish data! Hmm maybe Don can help… *RING RING* Hi Don, I need some upper Waimak fish observations for my model. Any ideas? Don Hmm, Ive got one dataset. Ill send it through… 10 minutes… To: Joe 01/02/09, 3.2, 3, 1 01/02/11, 3.1, 3, 1 10 minutes… *RING RING* Ok. Got the data. Where is the site located? Oh, its at laughing jack bridge. Coordinates? Ummm. (papers shuffle) What reference system?? I think its GDA94 Ok. What method is used? Umm, I think observations are done by Fyke net.. Oh…how does this compare to electric fishing? Umm DON? Joe Fish *CLICK* Modified From Peter Taylor (CSIRO)

The Interoperability Problem We want all information to be accessible. What does that mean? User can discover information (space, time, topic) enabled through well-defined interfaces. User can understand and assess information (relevance) for example enabled through well-defined vocabularies. User can access information through well-defined protocols. Notes:Users can be many men and robots! Information can come from many data sources

The Interoperability Problem Only information that is well managed is discoverable, understandable and accessible. Information management starts and information capture / generation (e.g. in the field). Federated information management and access is enabled through standards. We want all information to be accessible

What do we need to be able to federate biodata? Jochens thoughts

Towards a New Zealand interoperable biodata infrastructure - strawperson - Data Management Consistent Metadata Consistent Taxa descriptors Consistent reference information (geospatial, temporal) Consistent description of data (parameter, method, etc.) Delivery Agreed open standard web services

Information Management made consistent through a services stack - biodata - Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKESPI) DS2 (LAKESPI) DS3... DS3... Web Delivery Systems Dataset 1... Dataset 1... Metadata Services Metadata Services Taxonomic Information Services Observation & Measurement Names Services Procedures & Methods Information Services Consistency through Policy, Roles and Tools Consistency through Policy, Roles and Tools Standard Web Delivery Services Standard Web Delivery Services

How to manage data at NIWA? (excerpt) Describe metadata for your dataset according to the NIWA Metadata Management System. Describe observed variables / parameters / units according to the NIWA Measurement and Observation variables vocabulary. If you use a standard method (hint: many people / groups are using the method), ensure it is catalogued in the NIWA Procedures and Methods Register. Ensure the relevant taxa information is catalogued consistent with the NIWA Taxonomic Information Services. WHY? This will ensure the data is archived for the future, and discoverable, accessible and understandable by you your colleagues, and can be delivered through standard web interfaces.

Information Management made consistent through a services stack - biodata - Metadata Services Metadata Services Taxonomic Information Services Observation & Measurement Names Services Procedures & Methods Information Services Consistency through Policy, Roles and Tools Consistency through Policy, Roles and Tools Standard Web Delivery Services Standard Web Delivery Services Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKESPI) DS2 (LAKESPI) DS3... DS3... Web Delivery Dataset 1... Dataset 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1... Instutiutional Data Archives DS1 (FFDB) DS1 (FFDB) DS2 (LAKES PI) DS2 (LAKES PI) DS3... DS3... Web Delivery Datase t 1... Datase t 1...

NIWA EI Delivery Vision in New Zealand: Interoperability through open standards QA/Management CADDIS QA/Management Excel QA/Management FFDB Data Collection&QA Standards Data Archiving Standards Observations / Surveys / Data collection (to standards!) Data Archive NIWA (NEMO) Data Archive Auckland Council Data Archive Wellington Council Services Stack (Set of Standard Web Services) Data User NIWA Data User Council Data User Joe Public App/Portal ei.niwa.co.nz App/Portal NIWA QGIS App/Portal LAWA Multiple Uses Interoperability Data Delivery Standards Science User (R) App/Portal ALA App/Portal LakeSPI

Mission of TFBIS BSS Project Dataset a Dataset e Dataset c Dataset d Dataset b Data Sources different formats, different owners, different organisations user a user b user c user d user e ? Data compilation ad-hoc, manual, repeated, costly!! BSS standard formats and vocabularies, standard metadata, standard web service delivery Dataset a Dataset e Dataset c Dataset d Dataset b use case a use case b use case c TFBIS BSS project TFBIS BSS project Data Access: Portals, Web services, queries

BSS project- Milestones Clearly identify use cases (for occupancy data) and barriers and document this. Define required solutions and specify BSS infrastructure. Implement infrastructure for a limited set of organisations and data sources. Role-out across New Zealand, supported through RC, NEMS, MfE, Stats NZ, DOC, Councils, etc.

Data Access - a future situation Need fish data! To: Joe 01/02/09, 3.2, 3, 1 01/02/16, 3.1, 3, 1 Joe Fish To: Joe 01/02/09, 3.2, 3, 1 01/02/15, 3.1, 3, 1 To: Joe 01/02/09, 3.2, 3, 1 01/02/14, 3.1, 3, 1 To: Joe 01/02/09, 3.2, 3, 1 01/02/12, 3.1, 3, 1 01/02/09, 3.2, 3, 1 01/02/11, 3.1, 3, 1 01/02/14, 3.1, 3, 1 01/03/14, 3.1, 3, 1 01/10/15, 3.1, 3, 1 01/02/09, 3.2, 3, 1 01/02/11, 3.1, 3, 1 01/02/14, 3.1, 3, 1 01/03/14, 3.1, 3, 1 01/10/15, 3.1, 3, 1 Magic BSS Button Have fish data!

Current world of biodata sharing in NZ Data sitting in institutional or even private archives. Data (existence) is generally not published. Data is not discoverable. Data is not reconcilable. Lots of effort ($$) in pulling data together Data source a

Operations of the biodata services stack Standards Guidelines National tools (e.g. vocabulary services) BSS

A Common Biodata Infrastructure CBI Synonyms: Biodata Services Stack …

Federated bio observations data system What do we need? Use of (in all linked systems): Consistent geospatial / temporal context Consistent meta data (ANZLIC, ISO) Consistent taxa data (NZOR) _____________________________ These are harder… Consistent environmental feature vocabulary? (e.g. river names, lake names, …) Consistent (institutional?) methods vocabulary (use (national) standards as reference?) Consistent observations types vocabulary? (units vocabulary, parameter vocabulary)

A Common Biodata Infrastructure CBI Use-Cases / Why do we need it? Users: DOC, MfE, MPI, Primary Sector, Stats NZ, Regional Councils, CRIs, NGOs Uses: States/Trends (SoE), Tier 1 statistics, Research, ??? Data Providers: CRIs, DOC, Regional Councils, Te Papa, NGOs, Citizens Information Feedback (improved models, etc) Uses determine specification Can include: Reporting Tools Products / Analytical Tools Standards Data Management Tools Monitoring Tools

A Common Biodata Infrastructure CBI Information Types to be managed We are dealing with Sampling data – need vocabularies to ensure consistency Sampling data types: Survey Data (Systematic, well-defined scope) geo-chrono-taxa mapping Ad-hoc data: Collections, Citizen Science Vocabularies, e.g. Taxonomies (e.g. NZOR, soil classification) Includes metadata about entities in vocabularies (e.g. origin / occurrence) Descriptive Information about entities within a vocabulary Need to ensure contextual information can be integrated! E.g. climate, land cover etc.

Use case: Regional Councils - HBRC Tabulation of connected environmental parameters derived through interpolation/aggregation (rules required!) x/y, date/time, parameter1, parameter 2, … Tabulations of parameters x/y, date/time, parameter 1 Tabulations of parameters x/y, date/time, parameter 2 Survey 1 Survey 2 Metadata Standards Observation and Measurement Nomenclature Taxonomic vocabulary Standardized datasets Standardized formats / templates Authoritative Data Archive 1 Authoritative Data Archive 2 Data Archiving Data Collection Data Homogenization Data QA Data Delivery Data Connection / Amalgamation Survey x..