Presentation is loading. Please wait.

Presentation is loading. Please wait.

InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 1 InGeoCloudS INspired GEOdata CLOUD Services Linked Open Data.

Similar presentations


Presentation on theme: "InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 1 InGeoCloudS INspired GEOdata CLOUD Services Linked Open Data."— Presentation transcript:

1 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 1 InGeoCloudS INspired GEOdata CLOUD Services Linked Open Data for Data Services on the Cloud www.ingeoclouds.eu Dimitris Kotzinos University of Cergy Pontoise and FORTH-ICS This content by the InGeoCloudS consortium members is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Based on a work at http://www.ingeoclouds.eu/.the InGeoCloudS consortium membersCreative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License http://www.ingeoclouds.eu/

2 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 2 “Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone is doing it, so everyone claims they are doing it …” Dan Ariely, Professor, Duke University Big Data?

3 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 3 Design and build a cloud infrastructure for public agencies in the spatial-environmental field Provide an innovate (flexible,…) infrastructure for geo-data services “Move” public services to a cloud‐based infrastructure. Integrate geo-data by exploiting a Linked Data “model” The project wants to demonstrate that a Cloud infrastructure can be used by public organisations to provide more efficient, scalable and flexible services for creating, sharing and disseminating spatial environmental data ID-Card of the Project Objectives in a nutshell

4 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 4 ID-Card of the Project Partnership 5 Geological Surveys bringing in 6 initial Use Cases (datasets and applications) Ground Water Management Geo-Hazards GeoData Publication and Mapping 3 ICT organizations bringing key-expertise Cloud Computing Semantic Web and Linked Data GIS Software architecture and integration… EC Support

5 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 5 ID-Card of the Project Key Dates Feb 2012 March 2013 October 2013 July 2014 Pilot 1 We started Pilot 2 Regular Exploitation NOV.2013: Experts Workshop#2 MAY 2012: Experts Workshop#1

6 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 6 Fundamental scalable/elastic services for data management: Database Server, File Server, Linked Data Store Data publication modules An API: Web Services upon a REST-based architecture Data providers’ data and applications in the cloud Portal and Management Tools ID-Card of the Project Achievements

7 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 7 Infrastructure Description The Big Picture

8 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 8 Integrated Geo Applications

9 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 9 Map Design and Publication Service

10 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 10 From the map to the INSPIRE service Upload my dataset Create a new map (context) Create Layers (from dataset) Finalize the map Add functionali ties Publish the map Metadata Tag my dataset View, Download Defines my services Public Catalog Publish services

11 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 11 1 billion triples in the RDF triplestore published as Linked Open Data (LOD) Datasets from 3 different scientific fields Ground water (information and chemical analyses) (slow change) Landslides (fast change) Earthquakes (real-time measurements) Datasets from 4 different countries Denmark (Relational Data) France (Relational Data) Greece (text/xls data, XML data) Slovenia (Relational Data) ID-Card of the Project (Big?) Data and the 5 Vs Volume Variety Velocity Veracity X Value Later

12 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 12 Linked Data Services addR2RMLMappings tsv/csv xml json xml trig trix n3 … formats ldimport ldexport ldquery addXSLMappings geoldquery GeoJSON KML ShapeFile …. geoldtransform Inspire compliant inspire_export/ inspire_query_export

13 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 13

14 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 14 Querying Linked Data Store

15 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 15 Geoprocessing implemented as a WPS service: refers to ordinary kriging interpolation GeoProcessing WPS is provisioned in the ElasticWebServer component and uses parameters: Given by the user Fetched from InGeoCloudS Triplestore using LD-API

16 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 16 Monitoring and Costs More Big Data? A key needs in order to master one’s Cloud-based system. Two main levels in InGeoCloudS that help in better managing the cloud: Applications (analytics about usage) Resources (supervision of indicators, alerts and problems management)

17 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 17 Authorization Implementation Applications rely on a SSO token Applications check the user session and retrieve user profile Applications build permissions from the user roles To customize the behaviour of the application (e.g. mail on event), To control access (e.g. to some download services), To filter results displayed on the application GUI. Each software component of the platform control access to its own resources Different user types/roles Public (anonymous user) Registered user Data and Application provider Administrator

18 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 18 INGEOCLOUDS *aaS offerings IaaS Elements Servers Network Basic storage Administration APIs PaaS Elements Authentication Roles Workspaces, Portal RESTful APIs Basic Mapping Framework SaaS Elements Geo applications, Geo Publication, Catalogues Management, Opensearch Queries Geo-Processing DaaS Elements GSOM LOD APIs Data storage, import, sync Meta-data description,

19 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 19 User adoption Users are happy with what they have and they patiently wait until the problem surfaces Users with weak or no infrastructure are most receptive to turn to Cloud and/or Linked Data paradigms than others User see the cloud as a platform with more resources (memory, storage, processing) but they still want their applications to run there unchanged Security, privacy and trust Public vs. Private Clouds Publicly owned vs. Company-owned Clouds Challenges related to Big Data and the Cloud “Political” Challenges

20 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 20 Big Data storage No “native” RDF triplestore that exploits the Cloud capabilities Data security / privacy / control also a technical issue, especially when we deal with big data that you cannot monitor precisely Querying Big Data Fast querying for big data (many times the queries are simple but the volume is big) Support for geospatial queries (Fast) Indexing for (RDF) geospatial data “Exchanging” Big Data Data providers want to keep their own infrastructures and synchronize data between them and the cloud! Issues on synchronization and exchange of big data … Monitoring and Management on the Cloud More big data to manage? Challenges related to Big Data and the Cloud Scientific Challenges

21 InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 21 Thank you and Questions www.ingeoclouds.eu Dimitrios.Kotzinos@u-cergy.fr


Download ppt "InGeoCloudS, EDBT 2014 Special Session on «Big Data issues in the context of EU Projects » 1 InGeoCloudS INspired GEOdata CLOUD Services Linked Open Data."

Similar presentations


Ads by Google