1. Simulation Evaluation of Hybrid SRPT Policies
4/16/2017
Simulation Evaluation of Hybrid SRPT Policies
Mingwei Gong and Carey Williamson
Department of Computer Science
University of Calgary
April 19, 2004

2. Introduction Web: large-scale, client-server system
4/16/2017
Introduction
Web: large-scale, client-server system
WWW: World Wide Wait!
User-perceived Web response time involves:
Transmission time, propagation delay in network
Queueing delays at busy routers in the Internet
Delays caused by TCP protocol effects (e.g., handshake, slow start, packet loss, retransmits)
Queueing delays at the Web server itself, which may be servicing 100’s or 1000’s of concurrent requests
Our focus in this work: Web request scheduling
WORMS 2004

3. Example Scheduling Policies
4/16/2017
Example Scheduling Policies
FCFS: First Come First Serve
typical policy for single shared resource (“unfair”)
e.g., drive-thru restaurant; playoff tickets
PS: Processor Sharing
time-sharing a resource amongst J jobs
each job gets 1/J of the resources (equal, “fair”)
e.g., CPU; VM; multi-tasking; Apache Web server
SRPT: Shortest Remaining Processing Time
pre-emptive version of Shortest Job First (SJF)
give full resources to job that will complete quickest
e.g., ??? (express lanes in grocery store)(almost)
WORMS 2004

4. Research Methodology Trace-driven simulation Web server simulator
4/16/2017
Research Methodology
Trace-driven simulation
Input workload is empirical/synthetic trace
Web server simulator
Empirical trace (1 million requests, World Cup 1998)
Synthetic traces (WebTraff)
Probe-based sampling methodology
Based on PASTA: Poisson Arrivals See Time Averages
Any scheduling policy, any arrival process, any service time distribution.
The research on SRPT began in the 1960’s.
Previous work can be classified into three groups:
First, theoretical work proved that SRPT is optimal in terms of mean response time and mean slowdown compared with other scheduling policies.
Secondly, in practical work, SRPT was implemented in the Apache Web server
The experimental results are consistent with theoretical work.
Both works show that SRPT out-performs PS with respect to mean performance
However, The SRPT policy is rarely used in practice, the main concern is the fear of penalizing the large jobs.
WORMS 2004

5. Simulation Assumptions
4/16/2017
Simulation Assumptions
User requests are for static Web content
Server knows response size in advance
Network bandwidth is the bottleneck
All clients are in the same LAN environment
Ignores variations in network bandwidth and propagation delay
Fluid flow approximation: service time = response size
Ignores packetization issues
Ignores TCP protocol effects
Ignores network effects
(These are consistent with SRPT literature)
WORMS 2004

6. Performance Metrics Slowdown:
4/16/2017
Performance Metrics
Slowdown:
The slowdown of a job is its observed response time divided by the ideal response time if it were the only job in the system
Lower is better
We consider mean slowdown as well as the variance of slowdown (complete distribution)
WORMS 2004

7. Empirical Web Server Workload
4/16/2017
Empirical Web Server Workload
1998 WorldCup: Internet Traffic Archive:
Item
Value
Trace Duration
861 sec
Total Requests
1,000,000
Unique Documents
5,549
Total Transferred Bytes
3.3 GB
Smallest Transfer Size (bytes) 4
Largest Transfer Size (bytes)
2,891,887
Median Transfer Size (bytes)
889
Mean Transfer Size (bytes)
3,498
Standard Deviation (bytes)
18,815
WORMS 2004

8. Probe-based Sampling Algorithm
4/16/2017
Probe-based Sampling Algorithm
The algorithm is based on PASTA (Poisson Arrivals See Time Average) principle. S
Slowdown (1 sample) S
Repeat N
times S
WORMS 2004

9. Probe-based Sampling Algorithm
4/16/2017
Probe-based Sampling Algorithm
For scheduling policy S =(PS, SRPT, FCFS, LRPT, …) do
For load level U = (0.50, 0.80, 0.95) do
For probe job size J = (1B, 1KB, 10KB, 1MB...) do
For trial I = (1,2,3… N) do
Insert probe job at randomly chosen point;
Simulate Web server scheduling policy;
Compute and record slowdown value observed;
end of I;
Plot marginal distribution of slowdown results;
end of J;
end of U;
end of S;
WORMS 2004

10. Slowdown Profile Plot “asymptotic convergence” “crossover region”
(mystery hump) 8 PS x 1
1-p y
Slowdown
SRPT 1 8
Job Size
WORMS 2004

11. Notation Details Number of jobs in the system: J
Number of threads for a single server: K
Number of servers in the system: M
Probe jobs: 1KB, 10KB, 100KB, 1MB...
Number of probes: 3000
All simulation results are for 95% load
WORMS 2004

12. Single Server Scenario (M = 1)
PS: Processor Sharing
SRPT: Shortest Remaining Processing Time
FSP: Fair Sojourn Protocol
FSP computes the times at which jobs would complete under PS and then orders the jobs in terms of earliest PS completion times. FSP then devotes full service to the uncompleted job with the earliest PS completion time.
“FSP response time dominates PS” (i.e., is never worse)
E. Friedman and S. Henderson: “Fairness and Efficiency in Web Server Protocols”, Proc. ACM SIGMETRICS 2003.
WORMS 2004

13. Mean Slowdown (M = 1)
WORMS 2004

14. Variance of Slowdown (M = 1)
WORMS 2004

15. A Hybrid SRPT/PS Policy for a Single Server (M = 1)
Threshold-based policy, with threshold T
T-SRPT
Determining whether the system is "busy" or not depends on number of jobs (J) in the system.
If J <= T
Then use PS
Else use SRPT
Special cases: T = 0 is SRPT, T = is PS 8
WORMS 2004

16. Mean Slowdown for T-SRPT
WORMS 2004

17. Variance of Slowdown for T-SRPT
WORMS 2004

18. A Generalized SRPT Policy for a Multi-threaded Single Server
K-SRPT
Multi-threaded version of SRPT that allows up to K jobs (the K smallest RPT ones) to be in service concurrently (like PS), though with the same fixed aggregate service rate. Additional jobs (if any) in the system wait in the queue. Also preemptive, like SRPT.
Let s = min (J, K)
If J <= K
Then J jobs each receive 1/s
Else K jobs each receive 1/s (while J-K wait)
Special cases: K = 1 is SRPT, K = is PS 8
WORMS 2004

19. Mean Slowdown for K-SRPT
WORMS 2004

20. Variance of Slowdown for K-SRPT
WORMS 2004

21. Multi-Server Scenario
M-SRPT:
Let s = M
If J <= M
Then J jobs each receive 1/s (M-J idle servers)
Else M jobs each receive 1/s (while J-M wait)
M-PS:
Let s = max(M, J)
Each job receives a service rate of 1/s
M-FSP:
Let s = M
If J <= M
Then J jobs (under PS) each receive 1/s
Else M jobs (under PS) each receive 1/s
WORMS 2004

22. Mean Slowdown for M-SRPT
WORMS 2004

23. Variance of Slowdown for M-SRPT
WORMS 2004

24. Mean Slowdown for M-PS
WORMS 2004

25. Variance of Slowdown for M-PS
WORMS 2004

26. Mean Slowdown for M-FSP
WORMS 2004

27. Variance of Slowdown for M-FSP
WORMS 2004

28. Summary Slowdown profile plots for several policies
For the largest jobs, FSP better than SRPT and PS
For small jobs, FSP is sometimes worse than SRPT
Multi-threaded server results
T-SRPT and K-SRPT provide a smooth transition between SRPT and PS, implying smoother tradeoff in fairness between small jobs and large jobs
Multi-server results
With more servers, mean slowdown worsens, but variance of slowdown often improves
FSP does not response time dominate PS for M > 1
WORMS 2004

29. Thank You! Questions? For more information
4/16/2017
Thank You! Questions?
For more information
WORMS 2004