1. Predrag Buncic (CERN/PH-SFT) Virtualization R&D (WP9) Status Report

2. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 2 CernVM Project Portable Analysis Environment using Virtualization Technology (WP9)  Provide a complete, portable and easy to configure user environment for developing and running LHC data analysis locally and on the Grid independent of physical software and hardware platform (Linux, Windows, MacOS) Decouple application lifecycle from evolution of system infrastructure Reduce effort to install, maintain and keep up to date the experiment software People  Carlos Augado Sanchez (fellow, since 01/01/2008)  Leandro Franco (Marie Currie fellow, extended for 2 months to work on CVMFS)  Steffen Klemer (Summer student)  Artem Harutyunyan (visitor for 6 weeks, worked on adapting CernVM for ALICE)  Jacob Bloomer (Doctoral student, starting from 01/01/2009)  Predrag Buncic (PH/SFT 50%), Pere Mato (PH/SFT 5%) Experiment contacts  ATLAS (Paolo Calafura, Yushu Yao)  CMS (Benedikt Hagner, Marko Petek)  LHCb (Marko Clemencic, Hubert Degaudenzi)  ALICE (P.B.)

3. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 3 Current Status  Technology survey and initial prototyping carried out during first 3 months  Procured hardware and software licenses and setup a service and support infrastructure (web site, project and bug tracker)  Following the successful Kickoff Workshop in April, work plan redefined based on feedback from experiments Opted for frequent beta releases with constant interaction with experiments  Close interaction with experiments Bi-weekly steering meetings Regular reports to Architects Forum ATLAS is showing the biggest interest with LHCb and ALICE following CMS has recently found and committed manpower  Conferences and Talks White Area Lecture (CERN IT/GD) ACAT 2008 (plenary talk & poster)  First release ahead of the original schedule, delivered on time at the end of the October

4. How do we want to do that? Run exp. software locally Edit, compile, debug.. Submit jobs to Grid Retrieve and display results

5. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 5 Architecture “Thin” Virtual Software Appliance

6. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 6 Key Components rPath Linux 1 (www.rpath.org)  Slim Linux OS binary compatible with RH/SLC4 rAA - rPath Linux Appliance Agent  Web & XMLRPC User Interface rBulder  A tool to build VM images for various virtualization platforms  1 user licence acquired in April Substantial discount and no extra charges for support CVMFS - CernVM file system  Read only file system optimized for software distribution Aggressive caching  Operational in offline mode For as long as you stay within the cache  Derived from Open Source components of Parrot and GrowFS (www.cctools.org)www.cctools.org Adapted for CernVM by Leandro Franco Build types  Installable CD/DVD  Stub Image  Raw Filesystem Image  Netboot Image  Compressed Tar File  Demo CD/DVD (Live CD/DVD)  Raw Hard Disk Image  Vmware ® Virtual Appliance  Vmware ® ESX Server Virtual Appliance  Microsoft ® VHD Virtual Apliance  Xen Enterprise Virtual Appliance  Virtual Iron Virtual Appliance  Parallels Virtual Appliance  Amazon Machine Image  Update CD/DVD  Appliance Installable ISO

7. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 7 rAA Web UI

8. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 8 CernVM Components group-cernvm (core packages) group-cernvm (core packages) group-cernvm-devel (development tools) group-cernvm-devel (development tools) group-slc4 (SLC4 compatibility libs) group-slc4 (SLC4 compatibility libs) group- (groups and extra packages required by experiment) group- (groups and extra packages required by experiment) 100 MB compat-db4 compat-openssl compat-linstdc++slc3 compat-libxml2 compat-readline compat-tcl compat-tk group- -desktop (lightweight X environment) group- -desktop (lightweight X environment) group-cernvm-desktop X11 group-cernvm-desktop X11

9. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 9 Scalable Infrastructure 5 new servers (bought in August) 1 inherited from ALICE 2 retired from IT

10. Experiments are in control of their software

11. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 11 Application benchmarks http://cernvm.cern.ch/cernvm/index.cgi?page=BenchmarkResults

12. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 12 CernVM v.s. LXPLUS CernVM for ATLAS (benchmarks by Yushu Yao)  For the first time running, CernVM is 2-3 times slower, depending on the network speed, since it needs to cache files  Once the files are cached, the speed on CernVM is roughly the same as on LXPLUS.  CernVM is faster when reading large files (since they are local) Note: LXPLUS has a higher CPU rate than test computer (3GHz vs. 2.66GHz) Ratio to the execution time in LXPLUS

13. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 13 Developing CernVM JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC -Preparation Release 0.5 - Release 0.6 On time! Release 1.0 Available for download now! Kickoff Workshop - Release 0.8 - Release 0.7 - Release 0.91 (RC1) -Release 0.92 (RC2)

14. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 14 1.0 Release Available now for download from  http://rbuilder.cern.ch/project/cernvm/releases http://rbuilder.cern.ch/project/cernvm/releases Can be run on  Linux (KVM, Xen,VMware Player, VirtualBox)  Windows(WMware Player, VirtualBox)  Mac (Fusion, Parallels, VirtualBox) Release Notes http://cernvm.web.cern.ch/cernvm/index.cgi?page=ReleaseNotes http://cernvm.web.cern.ch/cernvm/index.cgi?page=ReleaseNotes HowTo http://cernvm.web.cern.ch/cernvm/?page=HowTo Appliance can be configured and used with ALICE, LHCb, ATLAS (and CMS) software frameworks

15. Download history 1.0 released 10 days ago

16. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 16 Primary (master) software repository Secondary software repository Secondary software repository Reverse Proxy Load Balancing Reverse Proxy Load Balancing Reverse Proxy CernVM (thin client) Site Reverse Proxy Load Balancing Reverse Proxy Regional Reverse Proxy Removing Single Point of Failure Content Delivery Network

17. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 17 Bridging Grids & Clouds BOINC  Open-source software for volunteer computing and grid computing  http://boinc.berkeley.edu/ http://boinc.berkeley.edu/  Summer student project in OpenLab based on LHC@HOME experience and CernVM image, aims to run ATLAS simulation using BOINC infrastructure  Successfully run ATLAS PanDA job in CernVM  CoPilot - an addition to CernVM to provide enable Pilot Jobs to run in VM AliEn,DIRAC,PanDA.. BOINC LHC@HOM E PanDA Pilot CoPilot

18. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 18 Mailing lists  cernvm-talk@cern.ch (Open list for announcements and discussion) cernvm.support@cern.ch (End-user support for the CernVM project) Savannah Portal  Please submit bugs and feature requests to Savannah at http://savannah.cern.ch/projects/cernvm CernVM Home Page:  http://cernvm.cern.ch http://cernvm.cern.ch rBuilder & Download Page:  http://rbuilder.cern.ch http://rbuilder.cern.ch CernVM Links…

19. 2008 Expenditure

20. 2008 Budget

21. CernVM - WP9 Status ReportPredrag.Buncic@cern,ch 11/11/2008 - 21 2008 Work Plan Remove single point of failure, develop and test Content Delivery Network  Doctoral student (J. Bloomer)  Compare different approaches and select the one most appropriate to our problem  Requires distributed testbed Can be realized as contributions from different labs participating in LHC experiments Alternative is to use emerging commercial Cloud infrastructure (Amazon) Migrate CernVM to rPath Linux 2 (SLC5 compatible)  Currently we are SLC4 compatible but experiments plan to move to SLC4 Migrate to VMware Infrastructure 3  Got 12 VI3 (free) licenses from VMware  Requires certified hardware SAN Continue collaboration with experiments  Improving UI and usability for ATLAS (2 months)  Work on developing CoPilot framework for ALICE (2 months) Initiate closer collaboration with IT  Replicating our pilot services and running them on IT hosted resources Consult and support other projects trying to use this technology  Geant4 Virtual Appliance for tutorials and testing  May require extra rBuilder license

22. 2009 Budget