1. CERN Openlab 21.06.2006 Openlab II virtualization developments Havard Bjerke

2. Physical machine Bread and butter testing Xen 2.0.7 Proof of concept GRID-in-a-box Complete LCG 2.6 installation Computing Element (CE) Storage Element (SE) Worker Node (WN) Successful deployment in LCG testbed Tests passed as good as native nodes Issues Automatic software updates cause /lib/tls to be restored Support for 2.4 kernels dropped

3. Virtualization in Batch Subsystems Client/CE Routing queue Execution queue [max 1 job] Execution queue [max 2 jobs] One-way WN SLC3 WN SLC4 Two-way WN SLC3 WN SLC4

4. VM Factory Image creation GRID filesystem / Storage element Worker node rpms, distribution, OS Caching, snapshots Deployment configuration, VM execution

5. VM Factory Worker node (Pre-) launch Distribution initialization scripts Execution Volumes RAM IP-address Gateway LCG gLite

6. VM Image management What does this involve? Creating distribution, installing software packages into the image Storing / caching the image somewhere Checkpointing / snapshots of VM state Creating VM images Few solutions exist Static library of images Dynamic creation of images, using RPM Dimitar: SLC3, SLC4 and Debian VM images HTTP interface Managing VM images Grid Programming Environment (GPE)

7. Deployment configuration Deploying the VM image on the target machine What does this involve? Managing disk volumes Transferring and managing the VM images Configuring parameters, initializing VM Provisioning resources Solutions Smartfrog Virtual Workspaces GPE

8. Virtual Workspaces

9. Grid Programming Environment Easy to deploy Grid application beans Services for Target systems Job management Storage management File transfer Integrates with Virtual Workspaces for resource provisioning Adds VM image management Interfaces with Globus

10. Xen expertise Links to virtualization community Xen development community Argonne National Labs Virtual Workspaces VM control service VM deployment configuration service Intel Grid Programming Environment (GPE) Image management Xen developers HP Smartfrog

11. State of the art hardware Openlab will continue to receive low number of early-version systems for testing. Examples: Woodcrest, Dempsey EM64T with VTx support Montecito IA64 with VTi support

12. Remove FUD Openlab does testing of virtualization to remove uncertainties CPU performance Cluster I/O ROOT Proof LCG

13. Conclusions Openlab is trying to stay on the leading edge of virtualization development & deployment Hopefully several areas are of interest to SA3 Openlab is willing to collaborate on concrete implementations, but the ownership would always have to be inside SA3 or other EGEE entities

14. CERN Openlab 21.06.2006 Smartfrog Xavier Grehant

15. ● a SmartFrog component, implementing the Prim interface ● Has attributes defined in the component description ● Lifecycle methods : ● sfDeploy, sfStart and sfTerminate ● Can look up other attributes through the component hierarchy ● an application : tree of Compound nodes and Prim leaves ● Implements the Composite design pattern (GoF) ● Several standard implementations of Compound are provided. An application can provide a custom implementation to deal with specific orchestration needs. ● The framework also permits component localization within the application. ● The SmartFrog language : application description ● Prototype-based object-oriented language (cf. Self) ● Convenient way of orchestrating components. Can also be done in the component code. SmartFrog is a Java framework that enables component deployment and management (orchestration) to build distributed applications.

16. Interdependent jobs Xen VM Prim Xen VM Prim AttributeHost 1 AttributeHost 2 AttributeHost 3 AttributeHost 4 Attributeflavour Attributeflavour Attributeconfig Attributeconfig VM dispatcher Compound Batch job Prim AttributeScripts AttributeLibs Resilient jobs manager Compound VM dispatcher Compound Batch job Prim AttributeScripts AttributeLibs Compound

17. Current Project : A SmartFrog application description for gLite middleware Applications: ● Automating updates of the middleware stack ● New versions currently take ~ 6 months to be adopted, as : ● Long-lived jobs have to terminate first. ● The whole stack has to be updated by hand on all machines ● (when not using something like Quattor) ● The downtime has to be scheduled. ● Could be solved by : ● Leveraging Xen VM migration to make the switch without downtime. ● Handling jobs dependencies during the update. ● Enabling whole gLite sites to be deployed on testbeds. ● Integration testing ● NMI : No control over the nodes while testing, so the whole site and test have to be deployed automatically.