Supported by EU projects 12/12/2013 Athens, Greece Open Data in Agriculture Hands-on with data infrastructures that can power your agricultural data products

Valeria Pesce Global Forum on Agricultural Research (GFAR) How to get datasets for agricultural sciences from the CIARD RING portal

Slide 3 of 63 Intro The CIARD RING is a global registry of information sources and datasets in agriculture -How can consumers of information find and retrieve datasets in the RING? -How can IT professionals get information on the level and mode of interoperability of the datasets and retrieve them from their applications? -How can producers and managers of information register their datasets so their information products can be found?

Slide 4 of 63 Objectives This presentation aims to: Explain how to search datasets in the RING Explain how other systems and applications can filter and get datasets from the RING automatically Explain how data managers can register and upload their datasets in the RING

Slide 5 of 63 The RING The CIARD RING is a global registry of information sources and datasets in agriculture, such as search engines, databases, repositories, Open Archives, directories, RSS feeds, open data endpoints… Why -Producers and managers of information need a place where their information products can be found -Consumers of information need to find suitable information sources -IT professionals need information on the level and mode of interoperability of information services

Slide 6 of 63 What is a dataset Wikipedia: A dataset (or data set) is a collection of data. (Narrow definition) Most commonly a dataset corresponds to the contents of a single database table, or a single statistical data matrix, where each column of the table represents a particular variable, and each row corresponds to a given member of the dataset in question. […] Nontabular datasets can take the form of marked up strings of characters, such as an XML file. W3C Government Linked Data Working Group (DCAT vocabulary): A collection of data, published or curated by a single source, and available for access or download in one or more formats.

Slide 7 of 63 Datasets in the RING Datasets can be registered as standalone data sources or as part of a “collection” (collection managers can register their data sources) A dataset is identified by: – uniform type of content – uniform criterion of grouping (thematic, geographic, institutional) – uniform data structure (dimensions / metadata set, encoding, reference value lists) One dataset can be made available / accessible as different “distributions” (format, protocol, URL) – E.g. a CSV file for download, or an XML file available as a feed

Slide 8 of 63 Faceted search of datasets in the RING

Slide 9 of 63 Some available facets

Slide 10 of 63 Getting a dataset

Slide 11 of 63 Getting a dataset Dataset Access Download or access through indicated protocol

Slide 12 of 63 How can other applications get the datasets test: DCAT vocabulary (W3C)

Slide 13 of 63 PREFIX rdf: PREFIX rdfs: PREFIX dc: PREFIX dcat: PREFIX adms: PREFIX doap: PREFIX skos: DESCRIBE ?dataset ?distro ?owner ?contact ?topic ?standard ?format ?protocol WHERE { ?dataset rdf:type dcat:Dataset. ?dataset dc:title ?title. ?dataset dcat:distribution ?distro. ?dataset dc:publisher ?owner. ?distro dcat:accessURL ?url. ?distro adms:representationTechnique. OPTIONAL { ?dataset doap:maintainer ?contact } OPTIONAL { ?dataset dcat:theme ?topic } OPTIONAL { ?distro dc:conformsTo ?standard } OPTIONAL { ?distro dc:format ?format } OPTIONAL { ?distro adms:representationTechnique ?protocol } } Run this query and get OAI-PMH datasets records URI that identifies the OAI-PMH protocol

Slide 14 of 63 Existing systems using RING datasets AGRISILRI SZTAKI Drupal module AgroKnow Drupal module

Slide 15 of 63 Processing the datasets? The RING is becoming also a registry of APIs to process datasets  Users will be able to search for APIs to perform the type of processing they need  They will get metadata about the technical specification of the APIs  In some cases, they will be able to launch the APIs over some selected RING datasets directly from the RING portal Examples of the first APIs that will be available will be agINFRA APIs for: Harvesting datasets Transforming datasets from one metadata set to another Automatically annotating dataset records with AGROVOC terms Enriching dataset records with VIVO and Google Scholar data Other workflows implemented in agINFRA

Slide 16 of 63 How to register a dataset You need to have a “dataset” that either: Is available at a URL (for download or through a protocol) Can be uploaded as a file to the RING Login and click on Add a service / dataset

Slide 17 of 63 How to register a dataset

Slide 18 of 63 How to register a dataset

Slide 19 of 63 How to register a dataset

Slide 20 of 63 How to register a dataset

Slide 21 of 63 How to register a dataset

Thank you! NameAffiliation