1. www.neresc.ac.uk A Dynamic Service Deployment Infrastructure for Grid Computing or Why it’s good to be Jobless Paul Watson School of Computing Science University of Newcastle, UK Paul Watson School of Computing Science University of Newcastle, UK Thanks: Chris Fowler, Charles Kubicek, Arijit Mukherjee, John Colquhoun, Savas Parastatidis, Mark Hewitt, Isi Mitrani, Jennie Palmer, Rob Smith, Paul McKee & Mike Fisher

2. www.neresc.ac.uk 2 Data in Science Bowker’s “Standard Scientific Model” 1 1.Collect data 2.Publish papers 3.Gradually loose the original data 1 The New Knowledge Economy and Science and Technology Policy, G.C. Bowker, E1-30-03-05

3. www.neresc.ac.uk 3 Publishing data as well as papers e-Science is trying to change this to: 1.Collect data 2.Publish data & papers e.g. SkyServer, OGSA-DAI publish databases through Web Services…

4. www.neresc.ac.uk 4 Problem: Moving Data Databases are good at localising computation & data But, often large amounts of data must still be transferred this may severely limit the performance

5. www.neresc.ac.uk 5 Jobs: the Grid Solution? Grid Computing offers remote job scheduling Therefore, we could package the analysis code & data as a job and send it to compute resources close to the data We decided to explore an alternative…

6. www.neresc.ac.uk 6 Why Jobs & Services? Grid applications are being built from Web Services But, if the computational requirements can’t be met by the service hosting environment then a job must be created and scheduled Why do we need both jobs and services? Dynasoar a service-only approach to building grid applications an infrastructure for the dynamic deployment of web services

7. www.neresc.ac.uk 7 Web Services

8. www.neresc.ac.uk 8 Dynasoar Components Web Service Provider (WSP) exposes service endpoints accepts the incoming SOAP message sent to the endpoint chooses a Host Provider and passes the message to it holds a copy of service code Host Provider (HP) manages computational resources (e.g. a cluster or a grid) accepts the message from the WSP dynamically deploys the service if necessary processes the message and returns any response Consumer

9. www.neresc.ac.uk 9 Routing to an Existing Service Deployment A request for s2 is routed to an existing deployment of the service

10. www.neresc.ac.uk 10 Dynamic service deployment R The deployed service remains in place and can be re-used - unlike job scheduling A request to s4 cannot be met by an existing deployment of the service

11. www.neresc.ac.uk 11 Dynasoar Advantages Simplicity: just services Efficiency: a deployed service can process many messages important if cost of deployment is high… e.g. VMs Support a range of new e-science/ e-business models: defining the interactions between the major components allows them to be distributed in a variety of ways

12. www.neresc.ac.uk 12 Dynamic Outsourcing Biocorp are experts in writing bioinformatics services They don’t want to manage their own compute resources Therefore, they use Hosting Inc to process messages sent to their services

13. www.neresc.ac.uk 13 The National Grid Service as a Host Provider A researcher writes their own services but does not have sufficient local compute resources They deploy a local WSP, and configure it so that it sends messages to the National Grid Service their services are then transparently deployed on the NGS as required

14. www.neresc.ac.uk 14 Brokers for Matching Web Service Providers to Host Providers Selection on: Price, Performance, Dependability,…

15. www.neresc.ac.uk 15 A Broker for e-Science Local Campus Grid National Grid Service

16. www.neresc.ac.uk 16 Moving Computation to Data The data owner provides compute resources close to a database Researchers can write services and deploy them on their own WSP The service is dynamically deployed close to the database when requests are sent to the WSP

17. www.neresc.ac.uk 17 Results for Deploying a Service Close to a Database

18. www.neresc.ac.uk 18 Tripartite Security Model The 3 actors can define policies (XACML) that Dynasoar enforces at run-time, e.g…. Accept only messages from WSPs trusted to not send malicious code Only use Host Providers trusted to not re-use the deployed service without payment Only send the message to a HP trusted not to look at the contents

19. www.neresc.ac.uk 19 Current Implementation GridShed Cluster Management (includes algorithm to decide when to deploy extra copies of a service to meet performance requirements)

20. www.neresc.ac.uk 20 New Host Provider Architecture Layer as high-level infrastructure over lower level grid fabric Use OMII Job Submission and Monitoring Service to provide stable interface to different underlying fabrics Newcastle Grid (Condor), National Grid Service, local clusters,….

21. www.neresc.ac.uk 21 Dynamic Service Grids Key to success: the availability of services for deployment

22. www.neresc.ac.uk 22 Active Information Repository

23. www.neresc.ac.uk 23 Current Work Experimenting with Bioinformatics Services Deploying Services in Virtual Machines can encapsulate a complex service implementation environment Use of QoS to guide decisions on where to deploy a service When and where to deploy within Host Provider? GridSHED project Reproducible e-science

24. www.neresc.ac.uk 24 Conclusions Grid applications can be built entirely from services jobless grid computing simpler conceptual model performance improvements due to sharing the cost of service deployment over multiple requests Dynasoar is built as a high-level infrastructure on top of existing grid fabrics Separating the Web Service Provider from the Host Provider – with a well-defined interface – opens up a range of e-science/ e-business models