1. EMI is partially funded by the European Commission under Grant Agreement RI-261611 Elisabetta Ronchieri, INFN CNAF Munich, 29 March 2012 WNoDeS Tutorial

2. EMI INFSO-RI-261611 Problem Statement WNoDeS Solution Deployment Use case WNoDeS System Administration WNoDeS User Information Future Work Conclusions Overview E. Ronchieri, EGI UF, 26-30 March 2012, Germany

3. EMI INFSO-RI-261611 PROBLEM STATEMENT E. Ronchieri, EGI UF, 26-30 March 2012, Germany

4. EMI INFSO-RI-261611 Large computing centers used for a wide range of applications by multiple user communities: – To define proper and timely configuration of compute resources without sacrificing efficiency, flexibility and security. User Communities: – To allow accessing resources by using a number of different interfaces, authenticating via several methods, exploiting existing local and distributed infrastructures. Requirements We Started With E. Ronchieri, EGI UF, 26-30 March 2012, Germany

5. EMI INFSO-RI-261611 To define a multi-layer framework to optimize the usage of computing center resources for multiple communities … starting from the INFN CNAF center (located in Bologna, Italy) that currently hosts about 10,000 computing cores, and 9 PB of disk space and 10 PB of tape space each day hosts about 80,000 jobs supports tens of international experiments of astro-particle physics (e.g., AMS2, Argo, Auger, Fermi/Glast, Magic, Pamela, Virgo) with about 15% of its computing resources supports the Italian Tier-1 for CERN-based LHC experiments of high energy physics (i.e, ATLAS, CMS, LHCb, Alice) and Tier- 0/1 for several others adopts Platform LSF as batch system and IBM GPFS as shared File System In short E. Ronchieri, EGI UF, 26-30 March 2012, Germany

6. EMI INFSO-RI-261611 WNoDeS SOLUTION E. Ronchieri, EGI UF, 26-30 March 2012, Germany

7. EMI INFSO-RI-261611 It is a software framework to integrate Grid and Cloud provisioning through virtualization. It is scalable and reliable – it is in production at several Italian centers, including the INFN Tier-1 (CNAF, Bologna) since November 2009. – Currently WNoDeS is managing about 2000 on‐demand Virtual Machines (VMs) there. It is a collaboration between INFN CNAF and IGI. On December 2011 WNoDeS was accepted in EMI and will be entered in EMI2 release. WNoDeS: Worker Nodes On Demands E. Ronchieri, EGI UF, 26-30 March 2012, Germany FeaturesReleased Batch Job ExecutionEMI2 Grid IntegrationEMI2 Cloud ComputingEMI2 update Virtual Interactive Pool (VIP)EMI2 update

8. EMI INFSO-RI-261611 Local batch jobs can be run on both real and virtual execution hosts (Worker Nodes). A virtual Worker Node is instantiated on-demand to provide computing resources for the execution of batch jobs just when needed. WNoDeS allows customized execution environments based on VMs, built and instantiated to match user requirements. Batch Job Execution: EMI2 E. Ronchieri, EGI UF, 26-30 March 2012, Germany

9. EMI INFSO-RI-261611 WNoDeS provides a seamless integration with Grid Computing EMI gLite middleware for transparent Grid job execution on Virtual Machines. Dynamic selection of resources (VM types, parameters) is done through standard EMI gLite Grid submission tools. Resource selection is based on a Glue 1.x schema attribute (SoftwareRunTimeEnvironment): This interoperates with current gLite WMS and CREAM CE middleware. Authentication is based on VOMS (Virtual Organization Membership Service). Grid Integration: EMI2 E. Ronchieri, EGI UF, 26-30 March 2012, Germany

10. EMI INFSO-RI-261611 WNoDeS has a Cloud Computing interface based on the OCCI API. This API is either accessible via CLI, or via a Web-based Application for ease of use. This provides on-demand computing resources allocated to users out of a common pool. Authentication is internally based on Short-Lived X.509 Certificates obtained through an online CA. However, users can authenticate through several methods, like Kerberos, Shibboleth, X.509/VOMS or Username/Password. Cloud Computing E. Ronchieri, EGI UF, 26-30 March 2012, Germany

11. EMI INFSO-RI-261611 WNoDeS VIP: on-demand provisioning of virtual user interfaces for local users out of a common resource pool. Based on a CLI, it can be easily used by computing center users seeking for additional, unused resources. Provides customized computing resources for interactive usage. for example, software development or physics analysis The provisioned virtual resource matches user requirements for RAM, CPU, bandwidth and shared file systems to be mounted. A caching mechanism is available, to reduce system provisioning time. VIP: Virtual Interactive Pool E. Ronchieri, EGI UF, 26-30 March 2012, Germany

12. EMI INFSO-RI-261611 WNoDeS Core Components E. Ronchieri, EGI UF, 26-30 March 2012, Germany EMI2 KVM

13. EMI INFSO-RI-261611 How WNoDeS Works E. Ronchieri, EGI UF, 26-30 March 2012, Germany WN

14. EMI INFSO-RI-261611 Mixed Mode feature, a new one included in EMI2, allows a farm to use the same physical WN to execute both real jobs and VM jobs. – This will increase the flexibility of the farm and allow better optimization of the resource usage. – There is no need to “reserve” WN to WNoDeS. From Grid to Cloud E. Ronchieri, EGI UF, 26-30 March 2012, Germany

15. EMI INFSO-RI-261611 How WNoDeS Works: Mixed Mode E. Ronchieri, EGI UF, 26-30 March 2012, Germany WN

16. EMI INFSO-RI-261611 WNoDeS Deployment E. Ronchieri, EGI UF, 26-30 March 2012, Germany

17. EMI INFSO-RI-261611 On top of LSF Batch System: The WNoDeS core version are installed in production at INFN Tier1. On top of open source batch system (Torque/Maui) The WNoDeS core version are installed in production at INFN Bari. Existing Deployment E. Ronchieri, EGI UF, 26-30 March 2012, Germany

18. EMI INFSO-RI-261611 USE CASE E. Ronchieri, EGI UF, 26-30 March 2012, Germany

19. EMI INFSO-RI-261611 Images requested by users / VOs should be agreed with system administrators. Currently this solution is strictly required for – Security reason: To avoid to modify the local logging; To avoid to unsecure services installed on the virtual machines (like a service that contains a known security bug). – Adopting local monitoring, accounting and security tools. – Customizing network and local storage configuration. Typical Use Case E. Ronchieri, EGI UF, 26-30 March 2012, Germany

20. EMI INFSO-RI-261611 WNoDeS System Administration E. Ronchieri, EGI UF, 26-30 March 2012, Germany

21. EMI INFSO-RI-261611 Operating System –WNoDeS works on SL5 and SL6 x86_64. DNS –Full access to DNS configuration is required. –Each virtual machine hostname needs to be added in the DNS. –Host resolution will be used by DHCP to provide ip address to the virtual resource. DHCP – Full access to DCHP configuration is required. – Each hostname associated to a virtual machine will be mapped with a mac address. KVM – It must be installed on all the real nodes. Repository – A dedicated repository to store all the virtual machine images need to be configured. – It can be a shared file system or a http repository. Batch System – It must be installed and configured (WNoDeS is tested against Torque/Maui and LSF). Installation Prerequisites E. Ronchieri, EGI UF, 26-30 March 2012, Germany

22. EMI INFSO-RI-261611 Nameserver It can be installed on any machine. Typically it does not contain hypervisor. Manager It can be installed on any machine. Hypervisor It needs to be installed on any machine where virtualization technology is enabled. Bait If mixed mode is enabled, it needs to be installed where hypervisor runs. If not, it needs to be installed on a virtual machine that will be instantiated by the hypervisor automatically. Site-Specific It needs to be installed on the machines where jobs are executed. It contains the pre-exec script and its configuration file: – If the FS is setup, they can be copied in a shared area; – If not, the component needs to be installed on any machine where jobs are executed and its configuration file needs to be changed in the same way. WNoDeS Installation E. Ronchieri, EGI UF, 26-30 March 2012, Germany

23. EMI INFSO-RI-261611 Version 2.0.0 RPMS: wnodes_nameserver wnodes_manager wnodes_bait wnodes_hypervisor wnodes_site_specific The exact version of the packages will be provided in the first EMI2 release. WNoDeS Packages E. Ronchieri, EGI UF, 26-30 March 2012, Germany

24. EMI INFSO-RI-261611 The system administration guide is now under verification [8]. It will be delivered in EMI2. It will contain also best practice not only configuration details. WNoDeS System Guide Administration E. Ronchieri, EGI UF, 26-30 March 2012, Germany

25. EMI INFSO-RI-261611 WNoDeS User Information E. Ronchieri, EGI UF, 26-30 March 2012, Germany

26. EMI INFSO-RI-261611 Prerequisite –Users MUST belong to a given VO (i.g., DTEAM) –Users MUST have a x.509 digital certificate How to submit user’s job to CE (i.g., cremino.cnaf.infn.it) directly as follows: –editing run_job.jdl file: Executable="run_job.sh"; Arguments=“arguments”; StdOutput="report_out.txt"; StdError="report_err.txt”; InputSandbox={"run_job.sh"}; OutputSandbox={"report_out.txt","report_err.txt"}; OutputSandboxBaseDestURI = "gsiftp://gftp2.ba.infn.it/lustre/cms/test_grid/”; –running commands: > voms-proxy-init --voms dteam > glite-ce-job-submit -a -r cremino.cnaf.infn.it:8443/cream-pbs-cloudtf run_job.jdl > glite-ce-job-status https://cremino.cnaf.infn.it:8443/CREAM422160619 Using Grid to provide VM: Case 1 E. Ronchieri, EGI UF, 26-30 March 2012, Germany

27. EMI INFSO-RI-261611 Prerequisite – Users MUST belong to a given VO (i.g., DTEAM) – Users MUST have a x.509 digital certificate How to submit user’s job to CE ( i.g., cremino.cnaf.infn.it ) as follows: – editing run_job.sh file: #!/bin/bash hostname date sleep 20 printenv – editing run_job.jdl file: Executable="run_job.sh"; StdOutput="report_out.txt"; StdError="report_err.txt"; InputSandbox={"run_job.sh"}; OutputSandbox={"report_out.txt","report_err.txt"}; requirements = other.GlueCEPolicyMaxWallClockTime > 100 && RegExp(".*cremino.*cloudtf",other.GlueCEUniqueID); Rank= ( other.GlueCEStateWaitingJobs == 0 ? other.GlueCEStateFreeCPUs : - other.GlueCEStateWaitingJobs) ; – running commands: >voms-proxy-init --voms dteam >glite-wms-job-status -a -e https://egee-rb-02.cnaf.infn.it:7443/glite_wms_wmproxy_server job.jdlglite-ce- job-submit -a -r cremino.cnaf.infn.it:8443/cream-pbs-cloudtf run_job.jdl Using Grid to provide VM: Case 2 E. Ronchieri, EGI UF, 26-30 March 2012, Germany

28. EMI INFSO-RI-261611 Prerequisite: – BDII needs to include GlueHostApplicationSoftware­Run­Time­Environment attribute to publish VM information How to obtain VM information – For example, a user that belongs to the VO DTEAM can get information about Tags configured on the queue cremino.cnaf.infn.it:8443/cream-pbs-cloudtf as follows: $ lcg-info --vo dteam --list-ce --query 'CE=cremino.cnaf.infn.it:8443/cream-pbs-cloudtf' --attrs Tag - CE: cremino.cnaf.infn.it:8443/cream-pbs-cloudtf - Tag CNAF GLITE-3_0_0 GLITE-3_1_0 GLITE-3_2_0 SF00MeanPerCPU_951 SI00MeanPerCPU_1039 VO-dteam-ScientificLinuxSLrelease5.7 Using BDII to publish VM info E. Ronchieri, EGI UF, 26-30 March 2012, Germany

29. EMI INFSO-RI-261611 FUTURE WORK E. Ronchieri, EGI UF, 26-30 March 2012, Germany

30. EMI INFSO-RI-261611 Existing WNoDeS features will be integrated in the WNoDeS version for EMI: Cache Manager WNoDeS Cloud CLI and Web Application VIP Virtual Network Configuration System Guide [8] and Test Plan documents will be updated accordingly. Hypervisor <> KVM will be evaluated. WNoDeS Cloud Web Interface will be integrated in the IGI portal Future Work E. Ronchieri, EGI UF, 26-30 March 2012, Germany

31. EMI INFSO-RI-261611 CONCLUSIONS E. Ronchieri, EGI UF, 26-30 March 2012, Germany

32. EMI INFSO-RI-261611 The benefits are both for site administrators and end users. – Site administrator can use the same set of services (such as the batch system, CREAM CE, NAGIOS and so on) to manage the same pool of resources allocated dynamically to satisfy the user requests. – End users can adopt grid and cloud concepts depending on their applications and different use cases. It is not required to convert an entire farm to WNoDeS: it can coexist and share resources with a traditional computing cluster not using virtual machines. Conclusions E. Ronchieri, EGI UF, 26-30 March 2012, Germany

33. EMI INFSO-RI-261611 For further information and questions – WNoDeS mailing list, wnodes@lists.infn.it The WNoDeS Web site – http://web.infn.it/wnodes http://web.infn.it/wnodes Acknoledgements for this talk – D.Salomoni (INFN CNAF), A. Italiano (INFN CNAF), G. Dalla Torre (INFN CNAF), D. Andreotti (INFN CNAF), G. Donvito (INFN Bari) Information E. Ronchieri, EGI UF, 26-30 March 2012, Germany

34. EMI INFSO-RI-261611 [1] A. Italiano, et al, WNoDeS, a Tool for Integrated Grid/Cloud Aaccess and Computing Farm Virtualization, CHEP 2010, Taipei [2] D. Salomoni, et al, An Authentication Gateway for Integrated Grid and Cloud Access, CHEP 2010, Taipei [3] C. Grandi, et al, Virtual Pools for Interactive Analysis and Software Development through an Integrated Cloud Environment, CHEP 2010, Taipei [4] D. Salomoni, A. K. Calabrese Melcarne, A. Chierici, G. Dalla Torre, A. Italiano, “Performance improvements in a large scale virtualization system”, PoS(ISGC 2011 & OGF 31)049 [5] D. Salomoni, D. Andreotti, L. Cestari, G. Potena, P. Solagna, “A Web-based portal to access and manage WNoDeS Virtualized Cloud resources”, PoS(ISGC 2011 & OGF 31)054 [6] M. Bencivenni, et al, “A portal for an easy access to the IGI grid infrastructure”, poster 9 at EGI TF, 19-23 September 2011, Lyon, France [7] D. Salomoni, E. Ronchieri, “WNoDeS: Requirements for Virtualized Services”. [8] E. Ronchieri, “WNoDeS: System Administration Guide”. [9] E. Ronchieri, “WNoDeS Tutorial”, EGI UF, 29 March 2012, Germany, in “EMI site administrators and the trainer workshop” References E. Ronchieri, EGI UF, 26-30 March 2012, Germany