1. Working Spaces: Virtual Machines in the Grid Kate Keahey keahey@mcs.anl.gov Argonne National Laboratory Tim Freeman, Frank Siebenlist {tfreeman,franks}@mcs.anl.govfranks}@mcs.anl.gov

2. Globus World 200502/10/2005 Towards Realizing the Grid Vision l Quality of Service u Dynamically controlled enforcement of various qualities u Not just per-process enforcement l Quality of Life u Being able to find the right configuration on the Grid

3. Globus World 200502/10/2005 We Need a Workspace! l A configurable execution environment, container u Good isolation properties u Good enforcement potential u Customizable software configuration l Library signature, OS, maybe even 64/32-bit architectures l We need to be able to create, manage and deploy it u We need to be able to negotiate/renegotiate an environment shape with a VO u A broker would then be able to negotiate resource allocations and map these workspaces onto

4. Globus World 200502/10/2005 Virtual Machines (VMs) l VMs happen to have: u Good isolation properties l Generally enhanced security, audit forensics u Good enforcement potential u Customizable software configuration l Library signature, OS, maybe even 64/32-bit architectures u Serialization property l VM images (include RAM), can be copied u The ability to pause and resume computations l Allow migration l Common concern: u Overhead: application, startup, resource usage

5. Globus World 200502/10/2005 Virtual Machines Primer l Different types of virtual machines u Full virtualization (VMware) l Run multiple unmodified guest OSs u Para-virtualization (Xen, UML, Denali) l Run multiple guest OSs ported to a special architecture u Single OS image (Vserver) l Paper: From Sandbox to Playground: Dynamic Virtual Environments in the Grid, Grid 2004 Hardware Virtual Machine Monitor (VMM) Guest OS (Linux) Guest OS (NetBSD) Guest OS (Windows)

6. Globus World 200502/10/2005 The Need for Speed LXVU SPEC INT2000 (score) LXVU Linux build time (s) LXVU OSDB-OLTP (tup/s) LXVU SPEC WEB99 (score) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U) Paper: Xen and the Art of Virtualization, SOSP 2003

7. Globus World 200502/10/2005 TCP results LXVU Tx, MTU 1500 (Mbps) LXVU Rx, MTU 1500 (Mbps) LXVU Tx, MTU 500 (Mbps) LXVU Rx, MTU 500 (Mbps) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 TCP bandwidth on Linux (L), Xen (X), VMWare Workstation (V), and UML (U)

8. Globus World 200502/10/2005 Scalability LX 2 LX 4 LX 8 LX 16 0 200 400 600 800 1000 Simultaneous SPEC WEB99 Instances on Linux (L) and Xen(X)

9. Globus World 200502/10/2005 Other Concerns l License u Open source (Xen, UML) l Visible effects of open source community at work u Commercial (VMware) l Also, XenSource l Distribution/Installation u Para-virtualization requires kernel modifications l Yes, but … everything else stays the same l Xen is soon to be part of the Linux kernel, Fedora Core 4 (May 05), is in Debian unstable, unofficial: Gentoo, Mandrake and SUSE distributions u Privilege l Xen (root, patch kernel, domain 0 privileges setup) l VMware Workstation (root, installation only)

10. Globus World 200502/10/2005 New Technology, New Challenges l How can we leverage the benefits of VMs in Grid technology? l How efficient will be this new technology? l How can we ensure a secure environment under these new assumptions? l How well will it all work in practice? l What new scenarios will this enable? l How will it change Grid computing? l What new problems will it create?

11. Globus World 200502/10/2005 Integrating VMs into the Grid Architecture Client request VM EPR inspect and manage deploy & suspend use existing VM image Create VM image VM Factory VM Repository VM Manager create new VM image Resource VM start program

12. Globus World 200502/10/2005 Supporting Services l Factory u Creates VM images l Eventually it may have to support negotiation u Images may be created based on an already existing image l VM Repository u Access to state describing a VM u Allows inspection, management, termination, potentially renegotiation, etc. l VM Manager u Service deploying VMs on nodes u Operations: stage, start, pause, stop, checkin l Once deployed, jobs may be executed in the virtual machines in a variety of ways

13. Globus World 200502/10/2005 Workspace Structure l Elements of a workspace u Workspace description (meta-data) l EPR/name, category, state, etc., also hardware, network, software configuration, etc. u Workspace implementation (VM image) l Workspace types u Workspaces conforming to set configurations u Provenance of VM images l Workspace instances u Workspace meta-data contains a name u Instance equality u Copying operation: copy image, meta-data, create a new name u Signing instances

14. Globus World 200502/10/2005 Security: New Opportunities, New Problems l VMs introduce a new layer of trust u VM monitor needs to be trusted as a technology u Trusted computing l VMs can be serialized and transferred as data u The integrity of a VM image needs to be protected (signing) u Private information on a VM image need to be protected (encryption) l VM private key: should a VM be able to assert its identity? l Application private keys, security context l VMs can be migrated (source --- > target) u Trust management: a VM image may be moved between parties that dont trust each other l A popular problem u Key management: target VM needs to verify the integrity and identity of a VM image in ways acceptable to the client: key management u Security context has to be preserved or renegotiated

15. Globus World 200502/10/2005 Migrating Securely target VM image in transfer: Signed and encrypted VM vouched for by sourceVM vouched for by target reestablish security context

16. Globus World 200502/10/2005 Security: New Twist on Old Problems l Deployment problem u Protecting the VM from the world l VMs are only as secure as the software they run l Who maintains all those VMs? Local administrators would have to maintain too many images yet need to protect against vulnerabilities u Protecting the world from the VM l One could use ones privileges as root on a VM (for example to generate harmful network traffic) l Although audit works great by the time the damage is done it is too late! l Deployment solutions u VO certification and authorization l Certification Authority to certify VM image l Site policies may require VMs to conform to site policies u Detection: Intrusion Detection Systems u Actions l Restricting network traffic: putting the good guys in jail l Right to take action

17. Globus World 200502/10/2005 Job Startup l c) job startup using a pre-configured job l b) job startup using an unpaused VM l a) job startup using GRAM

18. Globus World 200502/10/2005 Things We Didnt Talk About l Security u Processing of encryption and signing l Moving images u Image size: starting at 1MB, more typically 200 MB and upwards u Image diff (Rosenblum) u Proximity of image as matching criterion (VMPlant) u Mounting partitions (general on-site assembly) l Scalability u Distribute processes among existing VMs u Lightweight VMs: Denali l Clusters u Currently in progress: work on virtual cluster, collaboration with the COD team at Duke

19. Globus World 200502/10/2005 How does it work in practice? l Recent project: combining VMs and Grids to create a platform for bioinformatics applications l Some of the conclusions: u Use of virtual machines can significantly broaden the resource base u Saves installation time l EMBOSS installation: ~45 minutes l Deploying a 2GB VM image: ~6.5 minutes l Peace of mind: priceless! u Enforcement capabilities l Depend on the implementation but are generally better than what we have now l SC04 poster: u Quality of Life in the Grids: VMs meet Bioinformatics Applications, D. Galron et al

20. Globus World 200502/10/2005 Virtual Playgrounds l Define a virtual Grid in terms of requirements u Virtual workspaces u Networking requirements, virtual network u Storage and other requirements l Provide mechanisms to create a Grid l Provide services for the deployment of such virtual playgrounds on real resources l Ephemeral Grids built for a special purpose: u Scientists getting up a Grid for the purposes of a specific experiment run u A scientific simulation that gets discarded or interrupted but can potentially be restored later

21. Globus World 200502/10/2005 Conclusions l For Grids to scale we need a way to create and manage remote environments in the dynamically and effortlessly l Virtual is the new Real! u VMs present a very compelling solution l Efficiency, flexibility, migration, etc. u …and introduce some new challenges l New services, different models of sharing, security, etc. l Watch out for emergent properties and behaviors! l We need to work with the VM community to fine-tune requirements and features u Open Source helps! l A growing role for Virtual Organizations l Policy, Policy, Policy… u Policy of resource owners, VOs, users… l Closer to the dream on seamlessly negotiating, provisioning, renegotiating… l Status: a GT4 prototype that we keep evolving!

22. Globus World 200502/10/2005 Related Efforts l Condor u ClassAds, glide-ins l In-Vigo (VMPlant) l Virtuoso (VNET) l VIOLIN (UML + private networks) l Cluster on Demand l Workspaces Project u www.mcs.anl.gov/workspace

23. Globus World 200502/10/2005 Credits l Actively working: u Tim Freeman u Frank Siebenlist u Xuehai Zhang l Past contributions: u Karl Doering (UCSD) u Daniel Galron (OSU)