1. On-Demand Virtual Workspaces: Quality of Life in the Grid Kate Keahey keahey@mcs.anl.gov Argonne National Laboratory

2. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware the Grid metaphor How do we store energy? How do we charge for energy? How do we reliably deliver energy? What happens if a power station fails? How do we ensure quality of service? What elements make for a safe and efficient power Grid? How do we make sure that supply meets demand?

3. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware computational Grids How do we store computing? How do we charge for computing? How do we reliably deliver cycles? What happens if a power station fails? How do we ensure quality of service? What elements make for a safe and efficient power Grid? How do we make sure that supply meets demand? What is the unit of resource usage? How can we manage different computing environments? How can we ensure that disk, CPUs, network are all available? How can we negotiate for computation? NCSA ANLCaltech SDSC Tera Grid Grid Middleware We need a vehicle that will enable us to use Grid resources as easily and intuitively as we use electrical power today

4. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware what is virtualization? Lets see whats available and adapt my problem to use it Here is the environment I need to solve my problem -- deploy it on the Grid Can we provide the middleware that will enable this change of approach?

5. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware virtual workspaces l Focus on execution environments l Two aspects of workspaces: u Environment definition: We get exactly the (software) environment me need on demand. u Resource allocation: Provision and guarantee all the resources the workspace needs to function correctly (CPU, memory, disk, bandwidth, availability), allowing for dynamic renegotiation to reflect changing requirements and conditions. l Environment and resource allocation are now independent Quality of Life Quality of Service

6. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware how can we implement VWs? l Configuring physical machines u Slow and invasive u Environments are hard to describe u Limited/none enforcement options u Using environment management tools l Virtual Machines u Fast to deploy, much less invasive u Environments are easy to describe u Bonus: isolation, serialize, redeploy, migrate

7. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware virtual machine primer Hardware Virtual Machine Monitor (VMM) / Hypervisor Guest OS (Linux) Guest OS (NetBSD) Guest OS (Windows) VM App Xen VMWare UML Denali etc. Paravirtualization makes the performance overhead very acceptable

8. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware virtualizing other elements of an environment l Virtual storage u Combining many distributed physical resources l Virtual networks u Namespace management l Virtual private networks, ViNE, virtuoso, VIOLIN u Quality of Service l Overlay networks l Toward Virtual Grids u Putting all these elements together

9. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware GT4 workspace service l The GT4 Virtual Workspace Service (VWS) allows an authorized client to deploy and manage workspaces on-demand. u GT4 WSRF front-end u Leverages multiple GT services u Currently implements workspaces as VMs l Uses the Xen VMM but others could also be used u Current release 1.2 (September, 06) u http://workspace.globus.org

10. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware workspace service backstage Pool node Trusted Computing Base (TCB) Image Node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node The workspace service has a WSRF frontend that allows users to deploy and manage virtual workspaces The VWS manages a set of nodes inside the TCB (typically a cluster). This is called the node pool. Each node must have a VMM (Xen) installed, along with the workspace backend (software that manages individual nodes) VM images are staged to a designated image node inside the TCB VWS Node VWS Service

11. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware Image Node deploying workspaces Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Workspace - Workspace metadata (with image location) - Deployment request VWS Service l Adapter-based implementation model u Transport adapters l Default scp, then gridftp u Control adapters l Default ssh l Deprecated: PBS, SLURM u VW deployment adapter l Xen l Previous versions: VMware

12. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware workspace request arguments l A workspace, composed of: u VM image u Workspace metadata l XML document l Includes deployment-independent information: u VMM and kernel requirements u NICs + IP configuratoin u VM image location u Need not change between deployments l Resource allocation u Specifies availability, memory, CPU%, disk u Changes during or between deployments

13. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware Image Node interacting with workspaces Pool node Trusted Computing Base (TCB) Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node The workspace service publishes information on each workspace as standard WSRF Resource Properties. Users can query those properties to find out information about their workspace (e.g. what IP the workspace was bound to) Users can interact directly with their workspaces the same way the would with a physical machine. VWS Service

14. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware workspace service interfaces Workspace Service Workspace Factory Service Create() Workspace Meta-data/Image Deployment Request inspect & manage notify Workspace Resource Instance authorize & instantiate Workspace Service Handles creation of workspaces. Also publishes information on what types of workspaces it can support Handles management of each created workspace (start, stop, pause, migrate, inspecting VW state,...) Resource Properties publish the assigned resource allocation, how VW was bound to metadata (e.g. IP address), duration, and state

15. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware status l Latest Release: 1.2 released 9/14 u Significant improvement over 1.1.1 u At least one more release planned by the end of the year to include C client and better IP handling among others l To be included in the next VDT release l VW is an incubator project in dev.globus u New governance model for Globus Toolkit u http://dev.globus.org http://dev.globus.org u All software released under Apache license 2.0

16. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware support And thats what we do to bugs!

17. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware applications: ESF www.opensciencegrid.org/esf

18. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware ESF: division of labor Paper: Division of Labor: Tools for Growth and Scalability of Grids, ICSOC 2006

19. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware no STAR applications: STAR no STAR STAR VWS GRAM STAR Provisioning STAR nodes on TeraPort (UC): demonstrated at SC06 show floor

20. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware are we there yet? l YES: we do have reliable infrastructure that can implement the basic virtualization scenario l NO: the basic scenario addresses about 10% of virtualization potential (on a good day) Yes. And No...

21. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware a chicken and egg problem ChickenEgg

22. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware meet the chicken l Overcoming Xenophobia u Hypervisor installations are invasive l We need flexible site resource management systems u Security: the cure or the disease? l On the whole the cure, but it is a new tool u Will it scale? l This is not a question that a simulation could answer! u We need more effort in this area l Commercial deployments are moving faster u Hosting services, Amazons EC2, others… u There are more incentives l Pioneering is hard! u OSG

23. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware meet the egg l Suppose you have this infrastructure deployed, now what? u Where would be iTunes without music? l Original idea: develop a library of VM images u Labor intensive u Images age l Assembly line approach u rPath: scientific appliances and rBuilder l Appliance = application + its environment u BCFG2: configuration management tool l Producing and managing images l How do we describe, indentify, and query to find the right image?

24. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware virtualizing clusters l How do we construct virtual clusters? l How do we deploy virtual clusters on hardware resources? (overcoming xenophobia) u The overhead should be invisible to the client u Can we take advantage of application-specific knowledge when we schedule VMs? u What scheduler logic is appropriate and needed for scheduling workspaces? l Papers u Virtual Clusters for Grid Communities, CCGrid06 u Overhead Matters: A Model for Virtual Resource Management, VTDC 2006 (in SC06)

25. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware toward virtual grids l Deploying workspaces across multiple sites u Remember the STAR application l Virtualizing multiple aspects of a Grid u Combining networking and storage l Use Case: Combining QoS on data movement and execution u We want to get rid of workspace staging! l These are good times to be a meta-scheduler!

26. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware details, details… l Looking down the road u Assume we have resolved the simple problem… l What if we succeed? u 100s of VMs per physical manchine u Name management, storage, etc. l On the bright side u There may also be pleasant surprises

27. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware conclusions l We live in exciting times! l Making progress is hard l We have useful infrastructure that is being used by projects today on a small scale -- we need to move to larger scales l There are still many open problems l We have work to do!

28. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware Virtualization Workshop l Virtualization Technology in Distributed Computing (VTDC) 2006 u Co-held with SC06 u http://workspace.globus.org/vtdc2006 http://workspace.globus.org/vtdc2006

29. Virtual Workspaces5th Meeting of Spanish Initiative in Grid Middleware credits l Workspace team u Tim Freeman u Borja Sotomayor l With guest appearances by: u Ian Foster, Elizeu Santos-Neto, Frank Siebenlist, and others