1. © 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice Injecting Realistic Burstiness to a Traditional Client-Server Benchmark Ningfang Mi College of William and Mary Giuliano Casale SAP Research Ludmila Cherkasova Hewlett-Packard Labs Evgenia Smirni College of William and Mary Presenter: Lucy Cherkasova

2. 2 International Conference on Autonomic Computing and Communications (ICAC) 2009 Origin of Burstiness Enterprise and Internet applications: Clients DB Server Front Server Web + Application Server HTTP request HTTP reply SQL query SQL reply Burstiness ?? Highly Correlated Arrivals ?

3. 3 International Conference on Autonomic Computing and Communications (ICAC) 2009 Client-Server Benchmark E.g., TPC-W ( On-line bookstore Web site) Exponentially distributed user think times Exponentially distributed user think times Clients DB Server Front Server Web + Application Server HTTP request HTTP reply SQL query SQL reply Burstiness ?? Highly Correlated Arrivals ?

4. 4 International Conference on Autonomic Computing and Communications (ICAC) 2009 Accounts for randomness and variability … but not for burstiness … but not for burstiness  Can we ignore burstiness in the arrival process? Typical Client-Server Benchmark BurstinessVariability Service time Request number

5. 5 International Conference on Autonomic Computing and Communications (ICAC) 2009 Why Need to Inject Burstiness? Burstiness impacts the performance of resource allocation mechanisms. Example: Session-based admission control (SBAC) −User session: sequence of transaction requests −Session is a unit of work −Typically, long sessions are “sales”. −Useful system throughput is the number of completed sessions −Admission controller admits/rejects sessions based on observed CPU utilization of the server (a combination of last measurement and some history). L. Cherkasova, P. Phaal. Session Based Admission Control: a Mechanism for Peak Load Management of Commercial Web Sites. IEEE J. TOC, June 2002.

6. 6 International Conference on Autonomic Computing and Communications (ICAC) 2009 SBAC Reject a new session when utilization is above the threshold Abort an accepted session when the server queue is full highly undesirable Front Server Web + Application Server DB Server New Client Arrival Requests from already accepted clients limited server queue

7. 7 International Conference on Autonomic Computing and Communications (ICAC) 2009 Impact of Burstiness We performed experiments for the same workload with different arrival patterns: non-bursty vs bursty Aborted ratio = aborted sessions/accepted sessions highly undesirable Queue SizeNon-burstyBursty 2500.04%11.37% 5120.00%6.28% 8000.00%2.50%

8. 8 International Conference on Autonomic Computing and Communications (ICAC) 2009 Why Need to Inject Burstiness? (2) Service level agreement (SLA) −support given response time guarantees for accepted sessions SLA of 1.2s can be supported for 98% of requests with queue size =250 for non-bursty traffic Only 90% of requests meet SLA=1.2s bursty traffic. Queue Size Response Time (s) Non-Bursty Bursty

9. 9 International Conference on Autonomic Computing and Communications (ICAC) 2009 Limitations of Standard TPC-W Think times are drawn randomly from the exponential distribution identical for all clients incompatible Exponential think times are incompatible with the notion of burstiness. Need to inject burstiness into user think times.

10. 10 International Conference on Autonomic Computing and Communications (ICAC) 2009 Our Methodology Basic Idea: modify the distribution of client think time to create bursty arrivals Markovian Arrival Process −Regulate the arrivals by using a 2-phase Markovian Arrival Process (MAP). MAPs are variations of popular On/OFF traffic models that can be easily shaped to create correlated inter- arrival times All clients share a MAP(2) to draw think times A new module for client-server benchmarks index of dispersion −Regulate the intensity of traffic surges by using the index of dispersion. A simple tunable knob of burstiness

11. 11 International Conference on Autonomic Computing and Communications (ICAC) 2009 Index of Dispersion (I) Popular burstiness index in networking Definition −SCV – the squared coefficient of variation (variance/mean 2 ) −ρ k – autocorrelation coefficients i.e., correlation of service times −Exponential: I = SCV = 1 variability burstiness BurstinessVariability Service time Request number

12. 12 International Conference on Autonomic Computing and Communications (ICAC) 2009 Markovian Arrival Process (MAP) variability temporal locality MAPs have ability to provide variability and temporal locality. We use a class of MAPs with two states only Normal Traffic λ long Traffic Surge λ short 2 states: λ short > λ long p l,s p s,l p s,s p l,l time Num. of arrivals p l,s, p s,l, p s,s, p l,s shape correlation

13. 13 International Conference on Autonomic Computing and Communications (ICAC) 2009 MAP Fitting Input −Estimated mean service demands at servers: E[D i ] −Mean user think time E[Z] −The pre-defined index of dispersion I Output −A MAP(2) to draw user think times

14. 14 International Conference on Autonomic Computing and Communications (ICAC) 2009 MAP Fitting (2) Key: determine (λ short, λ long, p l,s, p s,l ) Condition for traffic surge Condition for normal traffic Mean think time We use non-linear optimizer to search for such f and p s,l and find a MAP(2) to best match the predefined I Departure > Arrival Arrival > Departure the arrival rate is f times higher than the throughput of the system the arrival rate is f times slower for balanced system throughput Balancing the height and the width of the burst

15. 15 International Conference on Autonomic Computing and Communications (ICAC) 2009 Realistic values for Burstiness −What is the range of realistic values for defining burstiness via index of dispersion I ? Exponential: I = SCV = 1 Bursty: values of thousands, −e.g., FIFA World Cup 1998, one of the servers over 10 days, I = 6300

16. 16 International Conference on Autonomic Computing and Communications (ICAC) 2009 TPC-W Testbed On-line bookstore Web site Testbed: clients + front server + DB server −Constant number of emulated browsers (EBs) User session −sequence of transaction requests −think time (mean=7 sec) between two transaction requests 14 transactions types grouped in three mixes: −Browsing mix −Shopping mix −Ordering mix

17. 17 International Conference on Autonomic Computing and Communications (ICAC) 2009 Validation – Arrival Process Arrival clients to the system (front server) Think times drawn by a MAP(2) with I create the bursty conditions. Shopping Mix Non-bursty (I=1) Time (s) Number of active clients Bursty (I=4000) Time (s) Number of active clients

18. 18 International Conference on Autonomic Computing and Communications (ICAC) 2009 Validation – Utilization Distribution Shopping Mix Non-bursty (I=1) Bursty (I=4000) pdf Utilization (%) Front DB

19. 19 International Conference on Autonomic Computing and Communications (ICAC) 2009 Validation - Average Latency Browsing Mix Response time (ms) Shopping Mix Response time (ms)

20. 20 International Conference on Autonomic Computing and Communications (ICAC) 2009 Validation – Latency Distributions Browsing Mix CDF Shopping Mix Response time (ms) CDF 0.83 2.98 0.04 1.25

21. 21 International Conference on Autonomic Computing and Communications (ICAC) 2009 Conclusion Burstiness critical for autonomic system design −need representative benchmarks for system evaluation −need reproducible and controllable bursty workloads Traditional client-server benchmarks ignore burstiness in arrival flows −e.g., TPC-W with exponential think times Explicitly inject burstiness −a simple and tunable parameter: index of dispersion −can introduce different intensity of traffic surges http://www.cs.wm.edu/~ningfang/tpcw_codes/ Supported by NSF grants CNS-0720699 and CCF-08114171 and HPLabs gift.