Carnegie Mellon University Computer Science Department 1 CLASSIFYING SCHEDULING POLICIES WITH RESPECT TO HIGHER MOMENTS OF CONDITIONAL RESPONSE TIME Adam Wierman Mor Harchol-Balter

Carnegie Mellon University Computer Science Department 2 HOW SHOULD USER REQUESTS BE SCHEDULED? router web server database …

Carnegie Mellon University Computer Science Department 3 SMALL RESPONSE TIMES ARE GOOD [Dellart 1999] [Zhou and Zhou 1996] BUT PREDICTABLE SERVICE IS BETTER BUT PREDICTABLE SERVICE IS BETTER

Carnegie Mellon University Computer Science Department 4 Users know the size of their request, x From past experience they expect a certain response time, E[T(x)] Users become frustrated when T(x) is much longer than E[T(x)]. WHY PREDICTABILI TY? T(x) determines frustration, not T. Need to provide predictable T(x) for all x

Carnegie Mellon University Computer Science Department 5 HOW DOES SCHEDULING AFFECT PREDICTABILITY? M/GI/1 preempt resume continuous service distribution work conserving scheduling

Carnegie Mellon University Computer Science Department 6 HOW SHOULD WE MEASURE PREDICTABILITY? Variance of conditional response time, Var[T(x)], and higher moments Coefficient of variation of T(x) Tail behavior of T(x)

Carnegie Mellon University Computer Science Department 7 We want to be predictable for all x HOW CAN WE COMPARE HIGHER MOMENTS OF T(x) ACROSS x?

Carnegie Mellon University Computer Science Department 8 A policy P is fair if E[T(x)] P / x ≤ 1/(1-ρ) for all x. Otherwise, P is unfair. MEAN [Sigmetrics 2003] Metric: E[T(x)] P / x Criterion: 1 / (1-ρ) GENERALIZE FROM THE MEAN

Carnegie Mellon University Computer Science Department 9 METRIC: E[T(x)]/x To compare across x, we must normalize by the average growth rate

Carnegie Mellon University Computer Science Department 10 E[T(x)] GROWS LINEARLY Theorem: Let E[X 2 ]< ∞. Under all work conserving scheduling policies Equality holds for PSJF, LAS, SRPT, PLCFS, and PS. The normalized mean of a conditional busy period: E[B(x)] / x

Carnegie Mellon University Computer Science Department 11 CRITERIO N: 1/(1-p) The goal of a criterion is to distinguish different behaviors of E[T(x)] / x

Carnegie Mellon University Computer Science Department 12 CRITERIO N: 1/(1-p) The goal of a criterion is to distinguish different behaviors of E[T(x)] / x x Var[T(x)] / x FILL IN x E[T(x)] / x FAIRUNFAIR 1/(1-ρ)

Carnegie Mellon University Computer Science Department 13 MEAN [Sigmetrics 2003] Metric: E[T(x)] P / x E[T(x)] P grows (asymptotically) linearly under all policies Criterion: 1 / (1-ρ) - differentiates between distinct functional behaviors - min P max x E[T(x)] P / x = 1/(1-ρ) for unbounded distributions A policy P is fair if E[T(x)] P / x ≤ 1/(1-ρ) for all x. Otherwise, P is unfair.

Carnegie Mellon University Computer Science Department 14 HOW CAN WE COMPARE HIGHER MOMENTS ACROSS x?

Carnegie Mellon University Computer Science Department 15 GENERALIZING THE METRIC: VAR[T(x)] X Variance grows linearly in x

Carnegie Mellon University Computer Science Department 16 Var[T(x)] GROWS LINEARLY Theorem: Let E[X 3 ]< ∞. Under all work conserving scheduling policies Equality holds for PSJF, LAS, SRPT, PLCFS, and PS. The normalized variance of a conditional busy period: Var[B(x)] / x

Carnegie Mellon University Computer Science Department 17 VARIANCE Metric: Var[T(x)] / x Variance grows (asymptotically) linearly Criterion: λE[X 2 ] / (1-ρ) 3 ???

Carnegie Mellon University Computer Science Department 18 GENERALIZING THE METRIC: HIGHER MOMENTS Raw moments Central moments Cumulant moments X X

Carnegie Mellon University Computer Science Department 19 CUMULAN TS Cumulants are a descriptive statistic, similar to the moments. They can be found as a function of the moments: or from the log of the moment generating function:

Carnegie Mellon University Computer Science Department 20 ALL CUMULANTS GROW LINEARLY Theorem: Let E[X i ]<∞. Under all work conserving scheduling policies: Equality holds for PSJF, LAS, SRPT, PLCFS. The normalized i th moment of a conditional busy period

Carnegie Mellon University Computer Science Department 21 HIGHER MOMENTS Metric: Normalized cumulants All cumulants grow (asymptotically) linearly Criterion: λE[B i ] ??? The normalized moments of the conditional busy period ???

Carnegie Mellon University Computer Science Department 22 DOES THIS CRITERION WORK FOR Var[T(x)]/x?

Carnegie Mellon University Computer Science Department 23 DISTINCT BEHAVIO RS Asymptotic behavior identifies behavior for all x Var[T(x)] / x x PLCFS, PS, SRPT ρ low Var[T(x)] / x x LAS, PSJF, SRPT ρ high PREDICTABLE UNPREDICTABLE λE[X 2 ] / (1-ρ) 3

Carnegie Mellon University Computer Science Department 24 A policy P is predictable if Var[T(x)] P / x ≤ λE[X 2 ] / (1-ρ) 3 for all x. Otherwise, P is unpredictable. PREDICTABILI TY Metric: Var[T(x)] P / x Var[T(x)] P grows (asymptotically) linearly for all common preemptive policies. Criterion: λE[X 2 ] / (1-ρ) 3 - differentiates between distinct functional behaviors - we conjecture that min P max x Var[T(x)] P /x is λE[X 2 ] / (1-ρ) 3

Carnegie Mellon University Computer Science Department 25 HOW DO DIFFERENT PRIORITIZATION TECHNIQUES AFFECT PREDICTABILITY?

Carnegie Mellon University Computer Science Department 26 Always Predictable Sometimes Predictable Always Unpredictable PS PLCFS Preemptive original size based PSJF PLJF Remaining size based SRPT LRPT Non-preemptive SJF LJF FCFS Age based LAS FCFS

Carnegie Mellon University Computer Science Department 27 WRAPUP Variability of T(x) is key to user satisfaction Metrics for comparing moments of T(x) across x Criteria for distinguishing the behavior of moments of T(x) A classification of Var[T(x)] / x

Carnegie Mellon University Computer Science Department 28 Metric: E[T(x)] / x Criterion: 1 / (1-ρ) Always Fair Always Unfair Non-preemptive size based Remaining size based Preemptive original size based Non-preemptive non-size based Age based PS PLCFS Sometimes Fair Always Predictable Sometimes Predictable Always Unpredictable PS PLCFS Preemptive original size based Remaining size based Non-preemptive Age based Metric: Var[T(x)] / x Criterion: λE[X 2 ] / (1-ρ) 3 Metrics: κ i [T(x)] / x Criteria: λE[B i ] ?

Carnegie Mellon University Computer Science Department 29 CLASSIFYING SCHEDULING POLICIES WITH RESPECT TO HIGHER MOMENTS OF CONDITIONAL RESPONSE TIME Adam Wierman Mor Harchol-Balter

Carnegie Mellon University Computer Science Department 30 Predictability  QoS guarantees:  Let g(x) = k x i ANOTH ER MOTIVA TION This should be some constant like 5%

Carnegie Mellon University Computer Science Department 31 ANOTH ER MOTIVA TION Predictability  QoS guarantees: We want the smallest i that gives a constant bound on Var[T(x)] / x 2i i=0 i=½ i=1 Var[T(x)] / x 2i  ∞ T(x) and E[T(x)] grow linearly X X

Carnegie Mellon University Computer Science Department 32 PLCFS – PS – LAS – SRPT x Var[T(x)] / x Preemptive Last Come First Served PLCFS

Carnegie Mellon University Computer Science Department 33 PLCFS – PS – LAS – SRPT Processor Sharing x Var[T(x)] / x PLCFS PS

Carnegie Mellon University Computer Science Department 34 PLCFS – PS – LAS – SRPT Least Attained Service x Var[T(x)] / x Under an Exponential with ρ=0.8, this is <17% of the jobs Under an Exponential with ρ=0.8, this is less than a factor of 4.9 PLCFS LAS

Carnegie Mellon University Computer Science Department 35 PLCFS – PS – LAS – SRPT Shortest Remaining Processing Time x Var[T(x)] / x Under an Exponential with ρ=0.8, this is <2.8% of the jobs Under an Exponential with ρ=0.8, this is less than a factor of 2.9 PLCFS SRPT

Carnegie Mellon University Computer Science Department 36 PLCFS – PS – LAS – SRPT Shortest Remaining Processing Time x Var[T(x)] / x x Low load ρ<0.4 regardless of the distribution up to higher loads for more variable distributions High load for every distribution, high enough load causes a hump PLCFS SRPT

Carnegie Mellon University Computer Science Department 37 FOUR DISTINCT BEHAVIO RS 1. Behaviors are distinguished by the value of max Var[T(x)]/x 2. Size based prioritization causes non-monotonic behavior Var[T(x)] / x x PLCFS, PS, SRPT ρ low FCFS, LCFS x x SJF Var[T(x)] / x x LAS, PSJF, SRPT ρ high