1. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Improving Performance of Internet Services Through Reward-Driven Request Prioritization Alexander Totok and Vijay Karamcheti Computer Science Department New York University

2. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Web Server Overload Conditions uConsequences increased request response times some requests are dropped successful session throughput suffers dramatically client dissatisfaction reduced revenues uCurrent solutions work with static client identity session-based admission control (SBAC) service level agreements (SLA) service membership per-client history-based approach –looks at a clients previous visits to the web site

3. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Maximizing Profit Brought By Internet Services uService profit (reward) maximization shopping web site: number of items sold uIdea: assign higher execution priority to the requests, whose sessions are likely to bring more reward how to predict a sessions reward? uOur solution: Reward-Driven Request Prioritization (RDRP) predicts a session's activities by comparing requests seen in it with aggregated client behavior uses Bayesian inference analysis to dynamically compute request priority in real time contrasts with the per-client history-based approach

4. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Service Usage Profiles (Patterns) uSession structure: first-order Markov chain corresponds to a typical service usage profile (pattern) uShopping scenario for TPC-W application (web store selling books) Mostly Buyers profile – more buying activity Mostly Browsers profile – more browsing activity

5. IWQoS2006, New Haven, CT, June 19 – 21, 2006 What Information Does RDRP Use? uUser load structure: {Profile i }; {p i } – percentage of sessions belonging to Profile k on-line request profiling and clustering analysis [Menasce99] uRequest reward reward i : per request type specified by the service provider web shopping scenario: reward(addToCart)=1 uRelative request execution cost for prediction of future server resource consumption cost i : per request type – average request processing time fine-grained profiling of request execution uOnly request reward is specified by the service provider

6. IWQoS2006, New Haven, CT, June 19 – 21, 2006 cost_expected How Does The Algorithm Work? priority = reward_attained + reward_expected cost_incurred + Step4:

7. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Prototype Implementation in J2EE uRequest priority used to allocate threads and DB connections

8. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Evaluation uShopping scenario for TPC-W user load with two usage patterns: Mostly Buyers/Mostly Browsers new sessions: bursty arrival process (B-model [Wang02]) uTechniques compared default – FIFO prioritization session-based admission control (SBAC) per-client history-based approach –success depends on how well prediction of a sessions behavior works –model different correlation between sessions rewards and assigned priorities: c = 0 (coin flip) c = 0.25 c = 0.5 (good oracle) c = 0.75 (very good oracle) c = 1 (perfect oracle) Reward-Driven Request Prioritization (RDRP)

9. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Overload (135%): Reward similar The bigger the better

10. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Overload (135%): Response Times The smaller the better

11. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Underload (80%): Response Times The smaller the better

12. IWQoS2006, New Haven, CT, June 19 – 21, 2006 Discussion uMain distinguishing features of RDRP tries to predict future sessions reward is oriented towards session completion works in an abstract application-generic manner uMay also take into account differences in user think times helps to distinguish between different service usage profiles in the Bayesian inference analysis uWhat if clients do not show stable behavioral patterns?

13. IWQoS2006, New Haven, CT, June 19 – 21, 2006 file Thank You!