Fair Share Scheduling Ethan Bolker UMass-Boston BMC Software Yiping Ding BMC Software CMG2000 Orlando, Florida December 13, 2000

Dec 13, 2000Fair Share Scheduling2 Coming Attractions Scheduling for performance Fair share semantics Priority scheduling; conservation laws Predicting transaction response time Experimental evidence Hierarchical share allocation What you can take away with you

Dec 13, 2000Fair Share Scheduling3 Strategy Tell the story with pictures when I can Question/interrupt at any time, please If you want to preserve the drama, don’t read ahead

Dec 13, 2000Fair Share Scheduling4 References Conference proceedings (talk != paper) www/cs.umb.edu/~eb/goalmode Jeff Buzen Dan Keefe Oliver Chen Chris Thornley Acknowledgements Aaron Ball Tom Larard Anatoliy Rikun

Dec 13, 2000Fair Share Scheduling5 Administrator specifies performance goals –Response times: IBM OS/390 (not today) –Resource allocations (shares): Unix offerings from HP (PRM) IBM (WLM) Sun (SRM) Operating system dispatches jobs in an attempt to meet goals Scheduling for Performance

Dec 13, 2000Fair Share Scheduling6 Scheduling for Performance Performance Goals complex scheduling software OS query update report response time workload measure frequently fast computation analytic algorithms Model

Dec 13, 2000Fair Share Scheduling7 Modeling Real system –Complex, dynamic, frequent state changes –Hard to tease out cause and effect Model –Static snapshot, deals in averages and probabilities –Fast enlightening answers to “what if ” questions Abstraction helps you understand real system

Dec 13, 2000Fair Share Scheduling8 Shares Administrator specifies fractions of various system resources allocated to workloads: e.g. “Dataminer owns 30% of the CPU” Resource: CPU cycles, disk access, memory, network bandwidth, number of processes,... Workload: group of users, processes, applications, … Precise meanings depend on vendor’s implementation

Dec 13, 2000Fair Share Scheduling9 Workloads Batch –Jobs always ready to run (latent demand) –Known: job service time –Performance metric: throughput Transaction –Jobs arrive at random, queue for service –Known: job service time and throughput –Performance metric: response time

Dec 13, 2000Fair Share Scheduling10 CPU Shares Administrator gives workload w CPU share f w Normalize shares so that  w f w = 1 w gets fraction f w of CPU time slices when at least one of its jobs is ready for service Can it use more if competing workloads idle? No :think share = cap Yes : think share = guarantee

Dec 13, 2000Fair Share Scheduling11 Shares As Caps dedicated system share f batch wkl throughput f need f > u ! Good for accounting (sell fraction of web server) Available now from IBM, HP, soon from Sun Straightforward: transaction wkl utilization u response time r r(1  u)/(f  u)

Dec 13, 2000Fair Share Scheduling12 Shares As Guarantees Good for performance + economy (production and development share CPU) When all workloads are batch, guarantees are caps IBM and Sun report tests of this case –confirm predictions –validate OS implementation No one seems to have studied transaction workloads

Dec 13, 2000Fair Share Scheduling13 Guarantees for Transaction Workloads May have share < utilization (!) Large share is like high priority Each workload’s response time depends on utilizations and shares of all other workloads Our model predicts response times given shares

Dec 13, 2000Fair Share Scheduling14 There’s No Such Thing As a Free Lunch High priority workload response time can be low only because someone else’s is high Response time conservation law: Weighted average response time is constant, independent of priority scheduling scheme:  workloads w w r w = C For a uniprocessor, C = U/(1-U) For queueing theory wizards: C is queue length

15 Two Workloads (Uniprocessor Formulas) 1 r r 2 = U/(1-U) r 1 : workload 1 response time r 2 : workload 2 response time conservation law: (r 1, r 2 ) lies on the line

16 Two Workloads (Uniprocessor Formulas) 1 r 1  u 1 /(1- u 1 ) r 1 : workload 1 response time r 2 : workload 2 response time constraint resulting from workload 1

17 Two Workloads (Uniprocessor Formulas)  Workload 1 runs at high priority: V(1,2) = (s 1 /(1- u 1 ), s 2 /(1- u 1 )(1-U) ) 1 r 1  u 1 /(1- u 1 ) r 1 : workload 1 response time r 2 : workload 2 response time constraint resulting from workload 1

18 Two Workloads (Uniprocessor Formulas)   V(2,1) 2 r 2  u 2 /(1- u 2 ) r 1 : workload 1 response time r 2 : workload 2 response time 1 r r 2 = U/(1-U)

19 Two Workloads (Uniprocessor Formulas)   V(1,2) = (s 1 /(1- u 1 ), s 2 /(1- u 1 )(1-U) ) V(2,1) 2 r 2  u 2 /(1- u 2 ) achievable region R r 1 : workload 1 response time r 2 : workload 2 response time 1 r r 2 = U/(1-U) 1 r 1  u 1 /(1- u 1 )

Dec 13, 2000Fair Share Scheduling20 Three Workloads Response time vector (r 1, r 2, r 3 ) lies on plane 1 r r r 3 = C We know a constraint for each workload w: w r w  C w Conservation applies to each pair of wkls as well: 1 r r 2  C 12 Achievable region has one vertex for each priority ordering of workloads: 3! = 6 in all Hence its name: the permutahedron

Dec 13, 2000Fair Share Scheduling21 3! = 6 vertices (priority orders) = 6 edges (conservation constraints) Three Workload Permutahedron r2r2 r1r1 r3r3 V(2,1,3) 1 r r r 3 = C V(1,2,3)  

r 1 : wkl 1 response time r 2 : wkl 2 response time r 3 : wkl 3 response time Three Workload Benchmark IBM WLM u 1 = 0.15, u 2 = 0.2, u 3 = 0.4 vary f 1, f 2, f 3 subject to f 1 + f 2 + f 3 = 1, measure r 1, r 2, r 3

Four Workload Permutahedron 4! = 24 vertices (priority orders) = 14 facets (conservation constraints) Simplicial geometry and transportation polytopes, Trans. Amer. Math. Soc. 217 (1976) 138.

Dec 13, 2000Fair Share Scheduling24 Response Time Conservation Provable in model assuming –Random arrivals, varying service times –Queue discipline independent of job attributes (fifo, class based priority scheduling, …) Observable in our benchmarks Can fail: shortest job first minimizes average response time –printer queues –supermarket express checkout lines

Dec 13, 2000Fair Share Scheduling25 Predict Response Times From Shares Given shares f 1, f 2, f 3,… we want to know vector V = r 1, r 2, r 3,... of workload response times Assume response time conservation: V will be a point in the permutahedron What point?

Dec 13, 2000Fair Share Scheduling26 Predict Response Times From Shares (Two Workloads) Reasonable modeling assumption: f 1 = 1, f 2 = 0 means workload 1 runs at high priority For arbitrary shares: workload priority order is (1,2) with probability f 1 (2,1) with probability f 2 (probability = fraction of time) Compute average workload response time: r 1 = f 1  (wkl 1 response at high priority) + f 2  (wkl 1 response at low priority )

27 r 1 : workload 1 response time r 2 : workload 2 response time V(1,2) : f 1 =1, f 2 =0 V(2,1) : f 1 =0, f 2 =1   Predict Response Times From Shares  V = f 1 V(1,2) + f 2 V(2,1) f 1 f 2 in this picture f 1  2/3, f 2  1/3

28 Model Validation IBM WLM u 1 = 0.24, u 2 = 0.47 vary f 1, f 2 subject to f 1 + f 2 = 1, measure r 1, r 2

29 Conservation Confirmed conservation law

30 Heavy vs. Light Workloads steep shallow u 1 = 0.24, u 2 = 0.47; u 2  2 u 1

31 Glitch? strange WLM anomaly near f 1 = f 2 = 1/2 (reproducible)

Dec 13, 2000Fair Share Scheduling32 Shares vs. Utilizations Remember: when shares are just guarantees f < u is possible for transaction workloads Think: large share is like high priority Share allocations affect light workloads more than heavy ones (like priority) It makes sense to give an important light workload a large share

Dec 13, 2000Fair Share Scheduling Three workloads, each with utilization 0.32 jobs/second  1.0 seconds/job = 0.32 = 32% CPU 96% busy, so average (conserved) response time is 1.0/(1  0.96) = 25 seconds Individual workload average response times depend on shares Three Transaction Workloads ???

Dec 13, 2000Fair Share Scheduling Normalized f 3 = 0.20 means 20% of the time workload 3 (development) would be dispatched at highest priority Three Transaction Workloads During that time, workload priority order is (3,1,2) for 32/80 of the time, (3,2,1) for 48/80 Probability( priority order is 312 ) = 0.20  (32/80) = 0.08 sum

Dec 13, 2000Fair Share Scheduling35 Formulas in paper in proceedings Average predicted response time weighted by throughput 25 seconds (as expected) Hard to understand intuitively Software helps Three Transaction Workloads

Dec 13, 2000Fair Share Scheduling36 The Fair Share Applet Screen captures on last and next slides from Experiment with “what if” fair share modeling Watch a simulation Random virtual job generator for the simulation is the same one used to generate random real jobs for our benchmark studies

37 Three Transaction Workloads note change from 32%

38 Simulation jobs currently on run queue

Dec 13, 2000Fair Share Scheduling39 When the Model Fails Real CPU uses round robin scheduling to deliver time slices Short jobs never wait for long jobs to complete That resembles shortest job first, so response time conservation law fails At high utilization, simulation shows smaller response times than predicted by model Response time conservation law yields conservative predictions

Dec 13, 2000Fair Share Scheduling40 Share Allocation Hierarchy customer 1 share 0.4 customer 2 share 0.6 Workloads at leaves of tree Shares are relative fractions of parent Production gets 80% of resources Customer 1 gets 40% of that 80% Users in development share 20% Available for Sun/Solaris. IBM/Aix offers tiers. x x      x x      production share 0.8 development share 0.2

Dec 13, 2000Fair Share Scheduling41 Don’t Flatten the Tree! is not the same as: customer 1 share 0.4 production share 0.8 development share 0.2 customer 2 share 0.6 For transaction workloads, shares as guarantees: customer 1 share 0.4  0.8 customer 2 share 0.6  0.8 development share 0.2

Dec 13, 2000Fair Share Scheduling42 Response Time Predictions flat Response times computed using formulas in the paper, implemented in the fairshare applet. Why does customer1 do so much better with hierarchical allocation? customer1 customer2 development share response times tree /80

Dec 13, 2000Fair Share Scheduling43 Response Time Predictions flat Often customer1 competes just with development. When that happens he gets 80% of the CPU in tree mode, 32/(32+20)  60% in flat mode Customers do well when they pool shares to buy in bulk. customer1 customer2 development share response times tree /80

Dec 13, 2000Fair Share Scheduling44 Fair Share Scheduling Batch and transaction workloads behave quite differently (and often unintuitively) when shares are guarantees A model helps you understand share semantics Shares are not utilizations Shares resemble priorities Response time is conserved Don’t flatten trees Enjoy the applet

Fair Share Scheduling Ethan Bolker UMass-Boston BMC Software Yiping Ding BMC Software CMG2000 Orlando, Florida December 13, 2000