Fabio Pasian, Marco Molinaro, Giuliano Taffoni The European Open Science Cloud: an opportunity for the astro community Fabio Pasian, Marco Molinaro, Giuliano Taffoni INAF - Osservatorio Astronomico di Trieste, Italy “INAF Science Archives & the Big Data Challenge” Workshop, 19 June 2019

Computing and data resources Large Research Infrastructures (RIs) Long tail of Science Surveys, Key Programmes «Standard» Research Citizen Science EPO 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

European Open Science Cloud (EOSC) EOSC is the EU answer to the challenges of open science EOSC = “virtual environment for science and technology professionals to store, share and re-use data across disciplines and borders” The mechanism is to open up by default all scientific data produced by H2020 projects (will be available as open data through EOSC) EOSC is part of the European Open Cloud Initiative (2 G€ in H2020 + at least 4 G€ additional investment from EU member countries) Funding dedicated to use of EOSC for Research Infrastructures and science domains is coming through dedicated H2020/INFRAEOSC calls 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

EOSCpilot H2020 project, funded at €10 million, 1 Jan 2017 – 31 May 2019 33 partners, led by UK STFC Multi-domain Support the first phase in the development of the European Open Science Cloud (EOSC) by: engaging with a broad range of stakeholders establishing the governance framework for the EOSC developing a number of pilots that integrate services and infrastructures Small participation from astronomy, partners were INAF (services and interoperability + pilot: VisIVO) + ASTRON (pilot: LOFAR archive) 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

EOSC-hub H2020 project, funded at > €20 million, 1 Jan 2018 – 31 Dec 2020 100 partners, led by EGI.eu Multi-domain Objectives: 49 services ready for use (from EGI, EUDAT, Indigo-DataCloud H2020 projects) simplify access to a broad portfolio of products, resources and services provided by the major international organisations through an open service catalogue remove fragmentation of service provisioning Participation from astronomy, to test/validate the infrastructure: ASTRON (LOFAR) – No INAF [ but INFN, CINECA, INGV ] participation 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

ESCAPE H2020 project, funded at €16 million, 1 Feb 2019 – 31 July 2021 32 partners, led by CNRS  inherits from ASTERICS CTA, ELT, EST, FAIR, HL-LHC, JIVE, KM3NeT, SKA (Astro + Particle) Map the VO framework to EOSC Aim to include the VO-enabled archive services from Astro RIs into EOSC VO Registry in EOSC Portfolio of astronomy VO services Contribution to EOSC hybrid cloud Containerised domain-specific services Integration of Science Platform with HPC and HTC infrastructures INAF 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

Data lifecycle (from EOSC-hub) Astro RIs Access, Deposition & Sharing Data Management, Curation & Preservation Federation Services Discover & Reuse Processing & Analysis

Integration of astronomy VO data and services into the EOSC ESCAPE WP4 (CEVO) VO Data Access Layer & Data Model standards Astro RIs VOSpace Access, Deposition & Sharing VO application standards VO metadata standards Data Management, Curation & Preservation Federation Services Discover & Reuse VO Single-Sign-On VO Registry VO Semantics standards VO query languages VO format standards Processing & Analysis VO Web service standards EOSC IVOA standards Science platform VO access (NEW)

OBELICS meeting / Madrid A big step forward … ESCAPE Each Astro RI produces data at a continuous rate, and performs basic processing (calibration, geometric registration, etc.): for that purpose it is expected to have its own dedicated e-infrastructure (incl. archive) EOSC can provide users with data-computing interoperability, to allow processing on archived data at the archive location (data centre) within a uniform environment (cut-out service  catalogues  classification  Machine Learning  user-provided code  …) From interoperable data collections (IVOA standards) to an integrated system of services for data-intensive astronomy  science 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni) 14-16 Sep 2016 OBELICS meeting / Madrid 9

Need to find, access, interoperate, re-use computing and storage facilities Archive Archive Archive Need to find, access, interoperate, re-use Archive Virtual Observatory 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

Need to find, access, interoperate, re-use computing and storage facilities Archive Archive Archive EOSC Need to find, access, interoperate, re-use Archive Virtual Observatory 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

EOSCpilot stakeholders meeting This is a vision shared by many colleagues all over the world. E.g.: EGI ( EOSC) in Europe, CANFAR/ ComputeCanada have a similar approach and have worked on interoperability of clouds (through INAF-OATs/CADC). (David Schade, CADC, UN Open Universe Workshop, 22 Nov 2017) 28-29 Nov 2017 EOSCpilot stakeholders meeting

Conclusions RIs (large facilities/ big projects) need their own e-infrastructure Archives need to offer computing services and user storage ( EOSC) EOSC will be complementary to dedicated facilities for RIs  will cover needs from “standard research” to “long tail of science” EOSC interface with large facilities is easily achievable through VO compliance (e.g. “VO-first” approach in LSST) Merger of HTC and HPC  HPDA (but new approach for exascale!) ICDI coordinates Italian e-infrastructure, and research institutes, to have proper national weight within EOSC governance 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

Is INAF ready? INAF archives to be VO- and EOSC-compliant INAF archives to provide services requiring computing resources basic: harvesting metadata from each other’s archives simple: calibration on-the-fly, cut-out, … [dedicated server] complex (user-provided code): data analysis, classification, ML, … [(colocated) computing resources linked to EOSC] There is no free lunch! pay for the usage of non-INAF EOSC resources (<<€ wrt commercial clouds) contribute to EOSC (and ICDI) with appropriate resources (computing + storage + knowledge! to represent the needs of the Astro community) Needed: training + education + careers for data scientists 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)

Thank you for your attention fabio.pasian@inaf.it The European Commission has supported ASTERICS (grant no. 653477) and EOSCpilot (grant no. 739563), and is supporting ESCAPE (grant no. 824064), in the framework of the Horizon2020 research and innovation programme. 19 June 2019 “INAF Science Archives & the Big Data Challenge” Workshop – European Open Science Cloud (Pasian, Molinaro, Taffoni)