Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education Data Sharing Grant Agreement n CHAIN-REDS tools for the use of a Cloud Infrastructure Giuseppe LA ROCCA, INFN Catania - Italy Clouds for Research and Education Workshop, 27 June 2014, Rome - Italy

Outline  Some introductory concepts and driving considerations  Catania Science Gateway Framework’s overview  The Cloud & Grid Engine  The architecture of the JSAGA adaptor for OCCI  The MyCloud portlet for clouds orchestration  Summary and Conclusions Clouds for Research and Education Workshop –

Some barriers limit the e-Infrastructure adoption (*) The eResearch2020 report / Some barriers limit the e-Infrastructure adoption Clouds for Research and Education Workshop –

The “non-global” middleware Genesis II Interoperability and easiness of access are issues Clouds for Research and Education Workshop –

A very «cloudy» sky… Clouds for Research and Education Workshop –

Interoperability & Interoperation (source: Wikipedia)  According to ISO/IEC (Information Technology Vocabulary, Fundamental Terms), interoperability is: "The capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units“  In engineering, interoperation is: “The setup of ad hoc components and methods to make two or more systems work together as a combined system”  The adoption of standards => are the key for a long-term sustainability! Clouds for Research and Education Workshop –

 A Scientist can sign in on a Catania Science Gateway using his/her federated credentials, select an application from a menu and seamlessly execute it on HPC machines, Grids and Clouds  The fractions of executions on the three different platforms can be adjusted to simulate the need to “boost” the resources in case of temporary peaks of activity Two Scenarios …  The Cloud Tenant of a real or virtual organisation can sign in on a Catania Science Gateway using his/her federated credentials, select virtual machine(s) from a geographically shared repository and deploy/move/copy it/them across his/her personal cloud  The graphic user interface will be very intuitive including point & click and drag & drop functionalities  The virtual machine(s) will belong to the same domain name (chain- project.eu in the particular case) independently of the site where it/they will be instantiated and of the underlying Cloud middleware stack Clouds for Research and Education Workshop –

Science Gateway Science Gateway Administrator Scientist Cloud tenant Users belonging to Identity Federations Users belonging to Identity Federations Grid/Cloud Engine (based on SAGA) Orchestrator (based on OCCI) Cloud #2 Cloud #n Cloud #1 Single domain HPC Clusters MyCloud / The Catania Science Gateway Framework’s high-level architecture Clouds for Research and Education Workshop –

The Catania Grid & Cloud Engine Users Tracking DB Science GW Interface SAGA/JSAGA API Job Engine Data Engine Users Track & Monit. Grid/Cloud/Local MWs User Tracking DB eToken Server 9 Clouds for Research and Education Workshop –

10 Use case implementation: JSAGA Adaptor for OCCI Science Gateway Catania Grid/Cloud Engine Security context Job management Stage-in/out 2) kill VM SG stage-in & run app VM1) stage-out SG VM SG start VM robot certificate Clouds for Research and Education Workshop –

CHAIN-REDS Science Gateway -CHAIN-REDS RI

Clouds for Research and Education Workshop – Enable advanced settings Specify the advanced settings for the application Start VM Select the VM profile Enable notification Select the VM template Job description CHAIN-REDS Science Gateway -CHAIN-REDS RI

 The Weather Research and Forecasting (WRF) Model is a next-generation meso-scale numerical weather prediction system designed to serve both atmospheric research and operational forecasting needs. (WRF) Running WRF on various distributed computing Infrastructures through the CHAIN-REDS SG CHAIN-REDS Science Gateway -CHAIN-REDS RI Clouds for Research and Education Workshop –

CHAIN-REDS Science Gateway -CHAIN-REDS RI Clouds for Research and Education Workshop –

15 Cloud 1 M/W 1’ Cloud 2 M/W 2’ Cloud 3 M/W 3’ Cloud 4 M/W 4’ Cloud 5 M/W 5’ Cloud 6 M/W 6’ Cloud 7 M/W 7’ Cloud 8 M/W 8’ Cloud 9 M/W 9’ Cloud 10 M/W 10’ MyCloud Clouds for Research and Education Workshop – CHAIN-REDS Science Gateway -CHAIN-REDS RI

16 10 clouds 6 countries 3 m/w stacks 1 SME IT ES CZIT ZA EG GR FedCloud Clouds for Research and Education Workshop – CHAIN-REDS Science Gateway -CHAIN-REDS RI

Managing VMs across different cloud providers: work! Current functionalities: Federated authentication Fine-grained authorisation Single/multi-deployment of VMs on a cloud and across clouds, including the EGI FedCloud Single/multi-move of VMs across clouds Single/multi-deletion of VMs on a cloud and across clouds SSH connection to VMs Direct web access to VMs hosting web services 17 Clouds for Research and Education Workshop – VMs are Moveable across Clouds Available VMs Multi/Single instance Available VMs Multi/Single instance

Managing VMs across different cloud providers: work! 18 Clouds for Research and Education Workshop – Turn On/Off the VM

 The Catania Science Gateway Framework successfully bridges e- Infrastructures built on different architectures (Grid, HPC, Cloud, local clusters, desktops, etc.) and make them interoperable at user application level thanks to standard adoption (SAGA, SAML, OCCI, etc.)  The MyCloud service allows seamless multi-cloud service operation across different OCCI-compliant middleware stacks  Next steps are:  Creation of the shared storage infrastructure to support stateful VMs  Allow deployed VMs to «find themselves» in MyCloud Summary and outlook Clouds for Research and Education Workshop –

Social Networks: Contacts Clouds for Research and Education Workshop –

Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education Data Sharing Grant Agreement n