QoS Aware Scheduling in a Cluster-Based Web Server Jiani Guo Architecture Lab Department of Computer Science and Engineering University of California, Riverside

2 Reference Performance Guarantees for Cluster- Based Internet Services, ICDCS 2003 Chang Li, Gang Peng, Kartik Gopalan Tzi-cker Chiuh State University of New York at Stony Brook

3 Web Services Courtesy: Performance Guarantees for Cluster-Based Internet Services, Chang Li.

4 Differentiated Service A system is said to be capable of affording differentiated service among service classes if The system permits its resources to be proportioned among the service classes Given sufficient request load, a service class receives at least as much resources as were assigned to it irrespective of the load on other service classes Resources not used by some service class may be distributed among other service classes. QoS Metrics The number of generic URL requests per second A generic URL request represents an average web site access which is assumed to take 10 msec of CPU time, 10 msec of disk channel usage time and 2000-bytes of network bandwidth For example: QoS requirement is 50 GRPS, which means 500 msec of CPU time, 500 msec of disk access time and 100 Kbytes of the network bandwidth

5 Scheduling Framework

6 Request scheduling Request selection Weighted round robin (WRR) No idea about the resource a request will consume on dispatching it  Predict per-request resource usage using history  Feedback to correct the prediction Server node selection Load balancing among server nodes (Least Load First) Select a node based on resource usage accounting What to account  CPU, disk and network bandwidth Accounting granularity  Per-request  Per-server  Per process-set Courtesy: Performance Guarantees for Cluster-Based Internet Services, Chang Li.

7 Performance Isolation

8 Performance deviation from ideal reservation Averaging Interval (secs)

9 My Previous Work on Scheduling: Scheduling Multimedia Jobs among Servers

10 Transcoding Workload A media unit is a Group Of Pictures(GOP) of MPEG stream A media unit can be transcoded independently by any Worker in the cluster. Transcoding one media unit is considered an independent job. No communication is required among jobs. Each job consumes similar amount of processing time. Consecutive media units in a stream are preferred to be processed in order.

11 Load Balancing Schemes How to take QoS into consideration? Streams make reservations Received service is proportional to the reservations Computing Server Scheduler Unit Buffer Retriever fetch a unit Find an available Computing Server Send the unit Media Server Computing Server

12 Framework of Fair Scheduling