1. 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG1 Do we have our heads in the cloud? Matthew J. Graham (Caltech, NVO) T HE US N ATIONAL V IRTUAL O BSERVATORY

2. The Cloud is… An elephant in a room of blind men: –SaaS –Utility computing –Web services –Platform as a service –Managed service provider –Service commerce platform –Internet integration All those things that the Grid promised but turned out not to be: –fast –scalable –resilient –available A virtualised notion of distributed computation, storage and communication over unlimited resources 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG2

3. Virtualisation Virtual workspaces: –An abstraction of an execution environment that can be made dynamically available to authorized clients by using well-defined protocols –resource quota (e.g. CPU, memory share) –software configuration (e.g. O/S, provided services) Implement on Virtual Machines (VMs): –abstraction of a physical host machine –hypervisor intercepts and emulates instructions from VMs and allows management of VMs –VMWare, Xen, etc. Provide infrastructure API –Plug-ins to hardware/support structures 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG3

4. Different types of cloud Amazon: –computing in the cloud –EC2 (with S3, SQS and SimpleDB) –Use Xen VMs –Images, Instances and Security Google: –Apps: Python module/API (with BigTable) –Working also with IBM (Blue Cloud) Microsoft: –Windows Live –Silverlight Yahoo: –Pipes –Working with Computational Research Laboratories (CRL) Sun –Project Hydrazine (JavaFX) Zoho Adobe (AIR) Commercial providers: –SalesForce –FlexiScale from XCalibre/Virtual Iron: cooler than EC2 Problem becomes availability of the right type of environment (hypervisor) 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG4

5. 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG5 Globus Workspace Abstracts hyperviser using WSRF Back-end implementation using Xen VMs on a variety of science clouds Metadata instance Deployment request Workspace Factory Service Workspace Service Workspace Resource Instance Deployment RP Inspect & manage Notify Create()Authorize & instantiate Contains GLOBUS

6. Clouds live in the Sky A lightweight abstraction layer a la VOSpace Resource oriented Data and services both treated as first-class entities Uses UWS interface for resource management 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG6

7. Identifiers and representations Use XRIs to identify all resources including computational results: xri://@!the.sky/myService/(+ ) Different representations (in RDF) returned dependent on resource aspect: 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG7 AspectMeaningRepresentation objectData objectVOSpace imageService imageVOResource instanceService instanceUWS metadata

8. Use case I have a data analysis service: ivo://nvo.caltech/MyService Upload bytes of a VM image: xri://@!the.sky/MyService/(+object) Create service instance: HTTP POST to http://@!the.sky/MyService/(+instance)http://@!the.sky/MyService/(+instance Returns xri://@!the.sky/MyService/1234 Commit instance to execution: HTTP POST to http://@!the.sky/MyService/1234/(+instance)/quotehttp://@!the.sky/MyService/1234/(+instance)/quote Define computation: xri://@!the.sky/MyService/1234+ data@xri://@!the.sky/MyData/myTable1+ params@xri://@!the.sky/MyData/myParam1 Link to results: xri://@!the.sky/MyData/myResults1 Retrieve bytes: xri://@!the.sky/MyData/myResults1/(+object) 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG8

9. Is this just blue sky thinking? Theres probably a future business in offering an abstraction layer between cloud computing services The missing element is cloud abstraction Proof-of-concept system based on VOSpace, UWS and a resource-oriented middleware layer in front of EC2 (and Google Apps) 20 May 2008IVOA Trieste: Grid & Web Services/AstroRG9