INFN OCCI implementation on Grid Infrastructure Michele Orrù INFN-CNAF OGF27, 13/10/ M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure

Outline Farm virtualization: WNoD The need for IaaS integration IaaS Application internal architecture IaaS Application flow of control Future works 2 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure

Virtualization within INFN INFN Tier-1 currently supports ca. 20 Virtual Organizations: each one has different software and resource requirements, and well-established workflows that should not be changed To satisfy their needs INFN has developed a solution called WNoD (Virtual Worker Nodes on Demand) With this kind of approach the installed software is not coupled anymore with physical hardware, and the resources are always dynamically shared thanks to the creation of VO-specific VMs at job execution time. 3 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Farm virtualization: WNoD

WNoD: high level design 4 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Farm virtualization: WNoD

Emerging Cloud requirements Usage of Grid resources in a flexible way, without the complex workflow that a typical grid job might have Third-party users (academia, industry, government) needing dynamic virtual execution environments INFN developers needing customized platforms to deploy and test their applications The idea is to keep the same LRMS that is already handling jobs from the Grid computing interface, and then expose a second service to manage requests of cloud-like resources. 5 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure The need for IaaS integration

Grid & Cloud: high level overview 6 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure The need for IaaS integration A mixed Grid infrastructure (both real and virtual worker nodes): only for job submission

Grid & Cloud: high level overview 7 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure The need for IaaS integration The same pool of resources is used to provide cloud-like services on the actual Grid infrastructure This allows to have homogeneous sharing with the resources used through the Grid interface Machines don’t need to be dedicated to either service

The IaaS layer We have implemented OGF's OCCI API as the interface of our application, due to its open and slim nature This solution works today, with a small subset of the OCCI API: the user requests a computing resource, gets an IP address back from the application and can then connect to the VM via SSH. The accounting and the authentication/authorization layers of the application will be completely integrated with the existing Grid infrastructure. Java Enterprise application built with industry leader technologies such as Spring, Hibernate, JPA, Restlet, Jetty, Maven 8 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure IaaS Application internal architecture

IaaS Application architecture 9 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure IaaS Application internal architecture We can fully leverage on unique Spring features such as Dependency Injection and Inversion of Control

OCCI implementation 10 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure IaaS Application internal architecture RESTful ws endpoints are defined in Spring’s application context XML files: following the “compute” descriptor. <lookup-method name="create" bean="computeResource"/> <bean name="computeResource" class="com.infn.cnaf.resources.compute.ComputeResource" scope="prototype">

OCCI implementation 11 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure IaaS Application internal architecture Operations are mapped to HTTP verbs with @Delete) Right now only plain-text format is supported on responses We are integrating ATOM and JSON public Representation storeRepresentation(Representation entity) { if (entity.getMediaType().equals(MediaType.APPLICATION_WWW_FORM, true)) public Representation toJson() {…}

Flow of control: create compute resource 12 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure IaaS Application flow of control 0/8

13 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 1/8

14 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 2/8

15 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 3/8

16 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 4/8

17 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 5/8

18 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 6/8

19 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 7/8

20 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Flow of control: create compute resource IaaS Application flow of control 8/8

On the way… Support for multi-core VMs UI improvements (Web based UI) Full compliance with the OCCI API Billing 21 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Future works

Questions??? 22 M.Orrù (INFN-CNAF) INFN OCCI implementation on Grid Infrastructure Questions?