1. Design and Evaluation of a Model for Multi-tiered Internet Applications Bhuvan Urgaonkar Internship project talk – Services Management Middleware Dept, IBM Aug. 20, 2004

2. Internet Data Centers Internet applications run on data centers Server farms Provide computational and storage resources Service-level agreements Response time guarantees Problem: Need good application models to determine right resource allocations

3. Multi-tiered Applications Internet applications: multiple tiers Example: 3 tiers: HTTP, J2EE app server, database Replicable components Example: clustered HTTP, J2EE server requests http J2EE database Load balancing gateway

4. Existing Application Models Several models for single-tier apps Queuing models for web servers: Chase et al (USITS 03), Chandra et al (IWQoS 2003) PODC 2004: G/G/1 based model Model only one (bottleneck) tier Ranjan et al (IWQoS 2002), Villela et al (IWQoS 2004)

5. Black-box Approach Black-box approach Treat application as a black-box Measure response time from outside Increase allocation if response time > SLA Use a model to decide how much to allocate Strawman #1: black-box for multi-tier apps Problems: Unclear which tier needs more capacity Bottleneck tier may not be replicable

6. Black-box Approach

7. Extension of Single-tier Model Strawman #2: use single-tier provisioning independently at each tier Example: Breakdown resp time into per-tier delays Use G/G/1 model for each tier Problems: How to breakdown resp time? G/G/1 based model found to be very conservative! Wasted capacity

8. Talk Outline Motivation Multi-tier Application Model Preliminary Evaluation Ongoing Work / Discussion Summary

9. Key Insights/Observations Tier i requests service from tier (i+1) Scheduling: PS closest tractable among policies Session-based workloads: A session consists of a succession of requests Think times are user dependent Quantity of interest: Per-request resp time requests http J2EE database Load balancing gateway

10. Queuing-theoretic Model Example: 3-tier application Natural model: Network of queues shown below... Capturing session-based workload Infinite server system Closed-queuing system X_1 X_2 X_3 Z

11. Mean-value Analysis MVA algorithm Inputs: Avg service times, visit ratios Computes avg. delays and resp. time... E[X_1] E[X_2] E[X_3] E[Z] V_0=1 V_1 V_2 V_3 Visit ratios

12. MVA Algorithm for m = 1 to M do Q m = 0 for n = 1 to N do begin for m = 1 to M do R m = D m for the infinite server R m = D m *(1+Q m ) for other servers X = n / Σ R m ……… throughput for m = 1 to M do Q m = X*R m …… Little’s Law end Key relation: A m (N) = Q m (N-1) A m : # customers an arriving customer finds in queue m Q m : # customers in queue m

13. MVA Algorithm: Discussion Can handle any service time distrib if scheduling discipline is PS Extension to multiple classes exists Need to measure service times and visit ratios on a per-class basis Gives only averages, not distribs Each queue is really only modeling a single resource

14. Finding Model Parameters Visit ratios Easy to obtain from various logs E.g. Apache-tomcat-mysql V_apache = 2 V_mysql = avg # queries per servlet V_tomcat = V_mysql + 1

15. Finding Model Parameters Service times Apache and Tomcat can be made to log time spent at and beyond them X_apache (T_apache–T_tomcat)/2 X_tomat (T_tomcat–V_mysql*X_mysql)/(V_mysql+1) X_mysql avg. query exec. time

16. What we haven’t captured … Inter-tier load balancers Resources held at tier i while awaiting response from tier (i+1) Increased service times at high loads E.g. context switches, protocol processing, contention for locks Tails of response times Multiple resources Load imbalances due to session affinity

17. Test Applications RUBiS (eBay like auction app) BrowseItems, PutBid, AuthorizeBid, PutComment, RegisterUser, SellItem, SearchItem, StoreBid, … RUBBoS (slashdot like b-board app) AcceptStory, BrowseStories, ModerateComment, PostComment, RegisterUser, RegisterStory, ViewStory, …

18. Experimental Setup Rubis (e-auctions), Rubbos (b-board) Apache, Tomcat, Mysql Apache mod_jk redirector Tiers 1 and 2 are replicable Java client Average think time = 1 sec One thread per session Apache+mod_jk Tomcat Mysql Client

19. RUBiS: Response Times Model works well in a restricted region Tomcat had a connection limit of 75

20. RUBiS: CPU Utilizations App tier is the bottleneck

21. RUBiS: Processes and Conns.

22. RUBBoS: Response Times Again, works well in a restricted region

23. RUBiS: DB-intensive workload Replaced SELECT with SELECT SQL_NOCACHE Database tier is bottleneck

24. Query Caching at the Database Able to capture effect of query caching at DB Interesting to do: Caching at app tier Reduced visit ratio at database

25. Multiple Classes of Sessions Class 1 : App server intensive Class 2 : Database intensive

26. Talk Outline Motivation Multi-tier Application Model Preliminary Evaluation Ongoing Work / Discussion Summary

27. Multiple Servers at a Tier Apache+mod_jk Tomcat Mysql Client Apache+mod_jk Tomcat Mysql Client

28. Load Imbalance rubis - 2 app servers 0 500 1000 1500 2000 2500 12510205080100 Num sessions Avg resp time (msec) Obs at apache Obs at tomcat1 Obs at tomcat2 Ongoing: Introduce a skew factor for adjusting the visit ratios to the servers in the replicated tier Session affinity Variable session requirements

29. Applying the Model to more Apps/Implementations EJB based implementations of RUBiS and RUBBoS TPC-W Workloads that stress resources other than CPU More??

30. Misc. Issues/Discussion Investigate utility of the model in Capacity planning Dynamic provisioning Admission control Measurements How many observations to gather? Handling incr. svc times at higher loads Context switching, locks, protocol processing, … Response time tails

31. Summary Network of queues based model Experimental evaluation for 2 apps Model works well in limited operating regions Simple enhancements to handle multiple classes, multiple servers, load imbalance More work needed on several aspects updating service times, how many observations, resp time tails, more apps and workloads

32. Acknowledgements Mike Spreitzer Asser Tantawi Giovanni Pacifici Thank you!

33. Extension of Single-tier Model