Flexible Provisioning of Service Workflows Sebastian Stein Supervisors: Nick Jennings Terry Payne KEG Seminar, Aston University 4 th March 2008

Flexible Provisioning of Service Workflows Agenda  Background & Motivation  Flexible Service Provisioning  On-Demand Invocation  Advance Agreements  Conclusions 2

Background & Motivation

Flexible Provisioning of Service Workflows 4 Background  Computer systems are increasingly distributed:  E-commerce Source: National Statistics Website,

Flexible Provisioning of Service Workflows 5 Background  Computer systems are increasingly distributed:  E-commerce  High performance computing

Flexible Provisioning of Service Workflows 6 Service-Oriented Computing  Distributed agents offer their capabilities as computer services, which are high-level behaviours that can be procured by consumers in order to achieve their objectives. These include:  Traditional business services (e.g., ordering components, making logistic arrangements, booking a flight ticket),  Computational services (e.g., data analysis, transformation and communication),  Information services (e.g., yellow pages, weather forecast, financial data).

Flexible Provisioning of Service Workflows 7 Workflows  Services are rarely used in isolation.  Usually, they form the building blocks for more complex applications.  Definition: A workflow is a collection of tasks and their dependencies.

Flexible Provisioning of Service Workflows 8 Taverna Workflow (myGrid) Source: Exploring Williams-Beuren Syndrome Using myGrid, Hannah Tipney,

Flexible Provisioning of Service Workflows 9 Pegasus Workflow Source: Pegasus Teragrid Talk SC2005 Seattle Washington,

Flexible Provisioning of Service Workflows Service Provisioning 10  Services are dynamically provisioned (selected) by consumers at run-time.  Services are provided by autonomous agents.  These may be unreliable (may fail or take longer than expected)…  …and heterogeneous. $ h -$20 -$10 -$5-$25 Failure! Value Deadline

Flexible Provisioning of Service Workflows 11 Problem Statement  How to design a service consuming agent able to deal effectively and efficiently with unreliable and heterogeneous service providers when executing complex workflows.

Flexible Provisioning of Service Workflows Related Work  Many current approaches concentrate on functional aspects of services and assume their behaviour to be deterministic.  Some work explicitly considers service failures:  Exception handling (e.g., fault handlers in WS-BPEL),  Fixed redundancy (e.g., replicated Web services),  Retry and timeout policies (Zeng 2005, Erradi 2006),  Non-functional service constraints (McIlraith and Son 2002).  These require significant manual input! 12

Flexible Provisioning of Service Workflows Related Work (Quality-of-Service Optimisation) 13  Local task QoS optimisation (Zeng 2004):  For each task, provision the provider that optimises some property for that task (e.g., cost, reliability, duration).  Global workflow QoS optimisation (Zeng 2003, Yu/Lin 2005):  Provision one provider for each task, so that a weighted sum of global performance characteristics is optimised:  Adaptive variants re-provision upon failure (Canfora 2005).  But: These do not reason explicitly about failures, rely on manually specified weights and constraints, and select single provider for each task.

Flexible Provisioning On-Demand Invocation

Flexible Provisioning of Service Workflows Central Idea  How to address uncertainty during provisioning? 15 Existing work mostly relies on single service for each workflow task. We can do better by exploiting parallel and serial redundancy. … and by taking into consideration service heterogeneity.

Flexible Provisioning of Service Workflows Service Model 16  We devised an abstract model to describe a service- oriented system.  Assumptions:  Assume silent “crash” failures.  Providers paid on invocation.  Failures and durations are independent.  Free disposal of redundant services (but cost still incurred!)  Utility function: Cost: c(s 1 ) = £100 Failure Prob.: f(s 1 ) = 0.01 Duration:

Flexible Provisioning of Service Workflows 17 Flexible Strategy  We want to find a provisioning allocation for each task, e.g.:  This is an optimisation problem: Expected reward Expected cost

Flexible Provisioning of Service Workflows Why is this difficult? 18  Intuitively,  Combinatorial problem:  Difficult objective function (probabilistic durations).  Based on this, we can show that provisioning is inherently hard...

Flexible Provisioning of Service Workflows Provisioning Provisioning Problem 19 Knapsack (NP-complete) PERT CDF (#P-complete)  Provisioning is NP-hard  Provisioning is #P-hard  Big problem as we wanted efficient methods for realistic workflows!

Flexible Provisioning of Service Workflows 20 Flexible Strategy  Approximate the expected utility of an allocation using a heuristic utility function:  Optimise this with local search. Estimated utility Success probability Estimated workflow duration pdf Estimated cost Reward function

Flexible Provisioning of Service Workflows Local Task Calculations  We start by calculating a number of performance parameters for each task in the workflow: 21 Success Probability: 95.00% Expected Cost: £30.00 Expected Duration: min Variance: min 2 Success Probability: 96.83% Expected Cost: £7.23 Expected Duration: min Variance: min 2 Cost:£1£30 Success:25%95% Duration:Exp (80) Gamma (10,6) 2 Service populations: Success Probability: 99.99% Expected Cost: £26.15 Expected Duration: min Variance: min 2

Flexible Provisioning of Service Workflows Global Workflow Calculations  These task parameters are then combined to estimate the overall expected profit: 22 Global Parameters: Success Probability: 68% Estimated Cost: £98.40 Estimated Duration: 132 min Variance: 912 min 2 100% 95% 80% 99% 100% 90% £24 £10 £3£42 £5 £ =

Flexible Provisioning of Service Workflows Empirical Evaluation  To test the strategy, we compare it to a number of benchmarks:  Naïve: Provisions a single provider for each task.  Models current approaches that do not consider service unreliability.  Global QoS: Optimises weighted QoS measures over entire workflow (set all w i =1/3, use maximum utility and zero reward time as budget/time constraints).  Adaptive Global QoS: As above, but also uses timeouts and re- provisions dynamically.  Local QoS: Optimises weighted QoS measure for each task. 23

Flexible Provisioning of Service Workflows Empirical Evaluation 24

Flexible Provisioning of Service Workflows Empirical Evaluation 25

Flexible Provisioning of Service Workflows Empirical Evaluation 26

Flexible Provisioning of Service Workflows Empirical Evaluation 27

Flexible Provisioning of Service Workflows Empirical Evaluation 28

Flexible Provisioning of Service Workflows Further Results  We can compare our performance to an optimal strategy for very small workflows (3 tasks!).  Achieves around 98% of optimal utility.  Results indicate that our strategy is robust to inaccurate information (with errors up to 10-15%). Beyond that, generally degrades gracefully, but problems when expected utility very low.  Trends hold on larger workflows (tested up to 1000 tasks). 29

Flexible Provisioning of Service Workflows So far…  We have proposed a flexible provisioning strategy that deals with uncertain service providers:  By provisioning multiple providers redundantly for critical tasks.  By re-provisioning services that seem to have failed.  By exploiting the heterogeneity of providers.  Our strategy outperforms the state of the art in flexible service provisioning.  But so far, our strategy:  Assumes that service populations are static throughout execution.  Assumes that services are always invoked on demand.  Does not adapt to new information during execution. 30

Flexible Provisioning Advance Agreements

Flexible Provisioning of Service Workflows Advance Provisioning  Increasingly, services will be offered in the context of pre- negotiated agreements (this is already emerging in computational Grids).  The agreements form a contract about when and how a service will be provided in the future. 32 I need service X in 2 hours. Reservation Cost:£20 Invocation Cost:£10 Start time: 2:00 Completion time: 2:30

Flexible Provisioning of Service Workflows Advance Provisioning  Performance characteristics might vary depending on time of provisioning (e.g., airline pricing policies): 33 Contract TermOn-Demand1h Advance12h Advance Cost£10£5£15 Duration45min20min10min Failure Probability10%2%0.1%

Flexible Provisioning of Service Workflows Modified System Model 34 Providers  Model a dynamic market:  Each time step:  Providers post offers, according to some stochastic process.  Consumer provisions offers.  Offers disappear (acquired by other consumers or withdrawn). Consumer Service Type: T 1 Start Time: 200 End Time: 220 Reservation Cost:£1 Execution Cost: £5 Penalty: £20 Failure Probability: 10% Defection Probability: 50%

Flexible Provisioning of Service Workflows Challenges  Future availability of offers uncertain.  Fixing advance agreements may mean that reservations costs are lost if preceding services fail.  Need to balance benefits of advance provisioning with risk! 35

Flexible Provisioning of Service Workflows Our Approach  Gradual Provisioning:  First make high-level provisioning decisions (how and when to provision tasks).  Follow these at run-time.  Adapt strategies when failures occur. 36 High-level decision Provisioned Completed Failure

Flexible Provisioning of Service Workflows High-Level Decisions  Assume we have a set of atomic provisioning strategies for each service type:  Performance statistics of strategies are learnt offline by observing the market. 37 Service Types Strategies … … Strategy w: Advance time Number of offers Selection strategy Expected performance: Reservation cost Execution cost Failure probability Duration (if successful) Duration (if failed) Variances of above

Flexible Provisioning of Service Workflows Contingency Planning  Atomic strategies represent single attempt at completing a task.  We can build simple plans from several such strategies to deal with failures: 38 Expected task performance: Success probability Reservation cost Execution cost Duration Variance

Flexible Provisioning of Service Workflows Overlapping Provisioning  Finally, associate a late probability p l with each task plan.  This indicates when services should be provisioned.  Higher p l results in less delays when provisioning in advance, but also increases probability that provisioned offers are lost when preceding tasks overrun.  Use heuristic based on critical path to estimate delays and to determine during which task to provision. 39 t x-2 t x-1 txtx txtx p l = 0.0 provision after t x-1 t x-2 t x-1 txtx txtx p l = 0.05 provision during t x-1 t x-2 t x-1 txtx txtx p l = 0.1 provision after t x-2

Flexible Provisioning of Service Workflows Strategy Summary  Given a high-level plan and late probability for each task, estimate utility in a similar manner as for on demand invocation, but include delays and reservation costs.  Optimise this using simulated annealing.  At run-time, follow task strategies, then incorporate information about provisioned offers and adapt strategy accordingly. 40

Flexible Provisioning of Service Workflows Empirical Evaluation  Small 8-task workflow with 5 service types.  Offer characteristics drawn from uniform distributions.  Comparison with three benchmark strategies:  Global QoS  Adaptive Global QoS  Local QoS  Also assume services always provide refunds for failures. 41

Flexible Provisioning of Service Workflows Empirical Evaluation 42

Flexible Provisioning of Service Workflows Empirical Evaluation 43

Flexible Provisioning of Service Workflows Empirical Evaluation 44

Flexible Provisioning of Service Workflows Empirical Evaluation 45

Flexible Provisioning of Service Workflows Conclusions  We proposed a novel algorithm that uses redundancy and dynamic re-provisioning to deal with uncertain service providers.  It does this in a flexible way by reasoning about service behaviours in the context of a decision-theoretic framework.  We first showed how it applies to scenarios where services are invoked on-demand, then extended it to environments with advance agreements.  In most scenarios considered, our strategy outperforms the state of the art in service provisioning. 46

Flexible Provisioning of Service Workflows Future Work  Improved prediction of workflow durations.  More expressive workflow models with branches and loops.  Consider more dynamic environments.  Incorporate meta-reasoning about time spent on optimisation. 47

Flexible Provisioning of Service Workflows Bibliography  Presented work from:  Stein, Jennings, Payne (2007). Provisioning Heterogeneous and Unreliable Providers for Service Workflows. In: AAAI-07. pp  Stein, Jennings, Payne (2008). Flexible Service Provisioning with Advance Agreements. In: AAMAS-08. (in press).  Related work on homogeneous providers:  Stein, Payne, Jennings (2008). Flexible Provisioning of Web Service Workflows. In: ACM Toit 8(4). (in press).  Other related work on QoS-based optimisation:  Zeng et al (2003). Quality driven web services composition. In: WWW-03. pp  Zeng et al (2004). QoS-Aware Middleware for Web Services Composition. In IEEE Soft. Eng. pp  Yu and Lin (2005). Service Selection Algorithms for Composing Complex Services with Multiple QoS Constraints. In: ICSOC-05.  Canfora et al (2005). QoS-Aware Replanning of Composite Web Services. In: ICWS-05. pp

Thank you! Any Questions? This work was sponsored by::