Predrag Buncic (CERN/PH-SFT) WP9 - Workshop Summary

CERN, 15/ Overview Summary of Talks in Virtualization Session To Do List Building the community Conclusions

CERN, 15/ Summary of Talks Intel (H Cornelius)  Multi and many core CPUs are not future, they are reality in Intel labs VMware (Richard Garsthagen)  Virtualization technology is much more than simple server virtualization  High Availability, Debugging, Vmotion tools, Power saving…  This extras do not come for free rPath (Tim Gerla)  rBuilder makes it really easy to build your own VM images  Independent of virtualization technology  rPath Appliance Platform makes it simple to upgrade, manage components and services in the appliance IgorFS (Thomas Fuhrman, TU Munich)  Using P2P technology to build fast and scalable file system for software distribution

CERN, 15/ CernVM: Initial Project Plan 1. Evaluation of the available virtualization technologies Understand and validate technologies by checking their performance in HEP environment 2. Evaluation of the tools to build and manage Virtual Appliances 3. Collect User Requirements from experiments Confront them with available technologies Suggest an optimal choice for a given use case 4. Development and deployment of a read-only distributed Network File System for software distribution Essential to keep the basic appliance small in size (<100MB) Validate performance, scalability and usability of such approach 5. Provide prototypes for at least two LHC experiments Assist experiments in adapting their software practices to this platform 6. Setup a service and support infrastructure

CERN, 15/ Recipe CernVM: Built using rBuilder

CERN, 15/ “Thin” Software Appliance JeOS (based on rPath Linux) rAArAA KERNEL fuse module FILESYSTEMFILESYSTEM rAA plugin Extra Liibs & Apps Cache HTTPDHTTPD Software Repository 10 GB1 GB0.1 GB LAN/WAN (HTTP)

CERN, 15/ Reusing existing tools App On same host: /opt/lcg -> /chirp/localhost/opt/lcg open(“/opt/lcg”) Parrot Chirp server On File Server /opt/lcg -> /grow/host/opt/lcg open(“/opt/lcg”) Cache Kernel NFSLFSFUSE Chirp Fuse !Cache Parrot is part of Cooperative Computing Tools

CERN, 15/ Foreseen development App On same host: /opt/lcg -> /chirp/localhost/opt/lcg open(“/opt/lcg”) On File Server /opt/lcg -> /grow/host/opt/lcg Cache Kernel NFSLFSFUSE CernVM Fuse !Cache

CERN, 15/ Alpha release 0.3 Available for download from  nvm-devel/releases nvm-devel/releases Can be run on  Linux (Xen,VMware Player)  Windows(WMware Player)  Mac (Fusion, Parallels) Minimal HowTO at m/?page=HowTo m/?page=HowTo Limited testing to verify that appliance can be configured and used with Alice and LHCb software frameworks ATLAS is next in line Waiting for input from CMS

CERN, 15/ Feedback from Experiments ALICE (Stefan Boettger):  HLT cluster  Currently 100, will grow to 900 cores  Running grid jobs on VMware VM when not busy with HLT  Concern: the price for VMvare server product line  Question: Can afford CERN site license? ATLAS (Yushu Yao):  Performance comparison of QEMU and VMWare  Conclusion that VMWare offers acceptable performance for casual user  Suggestion to distribute Atlas software as virtual disk that users can attach to the VM ATLAS (Amir Farbin):  Very welcome summary of user requirements based on two years of practical experience  Outlines some of the problems that we have tacked in CernVM prototype  Stresses concern about I/O performance and optimal way to move things in and out of VM

CERN, 15/ SFT (Jakob Blomer):  Benchmarking HEP applications in VM  Work done as summer student last year  Applications tested: Geant4, ROOT, AliRoot  Results point to very good performance of Xen under Linux  Surprisingly good performance of VMware and Parallels under MacOs IT (Andreas Unterkircher):  Using VMs in test environment  Deploying grid components for testing  VNODE tool  Xen supported by IT (Quattor profile for Xen exist for SLC4) SFT (Carlos Aguado Sanchez):  Looking for Open Source tools to deploy VMs on a small physical clusters  Enomalism2 chosen From CERN

CERN, 15/ To Do List Analyse Atlas requirements  Formalize in URD Continue development of read-only file system for software distribution  Compatible with both Atlas presentations  Simplify and make it faster  Add extra management interfaces Explicitly fetch subdirectory Pin the directory content in cache Clear the cache Continue work towards first release Add support for ATLAS, CMS Functional test of exp frameworks Performance test Add capability to act as Grid User interface  Verify that Grid Data Management works Know problem with rfio Documentation

CERN, 15/ Regular meetings (2 weeks) Mailing list Web site  Building the community

CERN, 15/ Conclusions The work on the project has started according to initial planning and is progressing well  Planning based on initial discussions with experiments trying to identify interest for collaboration  Update plan with items in To Do list and suggestions from the floor Contacts have been established with goal to carry out further research The evaluation of performance indicates that it is possible to use virtualization technologies in HEP context rBilder instance installed at CERN and seem to live up to the promises Work on setting up scalable service infrastructure is in progress Prototype of file system for s/w distribution is implemented Alpha release is available and is ready for 2 experiments  Work on Atlas will start next Expecting first release that could be used by the real users in September