FAIRNESS IN QUEUES Adam Wierman Carnegie Mellon University acw@cs. cmu FAIRNESS IN QUEUES Adam Wierman Carnegie Mellon University acw@cs.cmu.edu Joint work with Mor Harchol-Balter

“SCHEDULING SUCCESS STORIES” ARE EVERYWHERE Web Servers users Routers Internet Disks CPUs Locks Databases …also p2p, wireless, operating systems… Biersack, Rai, Urvoy-Keller, Harchol-Balter, Schroeder, Agrawal, Ganger, Petrou, Misra, Feng, Hu, Zhang, Mangharam, Sadowsky, Rawat, Dinda, McWherter, Ailamaki, & others

THE ESSENCE OF A “SCHEDULING SUCCESS STORY” Processor Sharing (PS) bottleneck resource web server, router, etc. Goal Minimize user response times Make sure to provide details of the examples routers and wireless PS at a flow level Make sure to say that SRPT is preemptive small arriving job interupts job in service. Make sure to say why SRPT is good (smalls don’t ges) Use a different scheduling policy

? WHAT POLICY SHOULD WE USE? SRPT WINS BIG SRPT ? Sched- PS FCFS uling load 0 0.25 0.5 0.75 mean response time PS SRPT FCFS What about fairness to large jobs? ?

ARE POLICIES THAT PRIORITIZE SMALL JOBS FAIR TO LARGE JOBS? What does “fair” mean? ? M/GI/1 Preempt-Resume Queue

WHAT IS FAIRNESS? Definition: A fair decision is one that makes the proper balance of conflicting interests The “proper balance” is determined by consulting the majority, employing logic, referring to the divine.

Which is more fair ? when two jobs of the same size arrive one afer the other... B 1 A 2 B A

Which is more fair ? when a small job and a large job arrive at the same time... B 1 A B 2 A

BUT, THE SETTING MATTERS OUR SETTING: Are the response times of large jobs “unfairly” long?

WHAT IS FAIRNESS? Definition: A fair decision is one that makes the proper balance of conflicting interests The “proper balance” is determined by consulting the majority, employing logic, referring to the divine.

EMPLOYING LOGIC Aristotle’s notion of fairness Like cases should be treated alike, different cases should be treated differently, and different cases should be treated differently in proportion to their differences. Response times should be proportional to job sizes. T(x)/x should be constant slowdown

EMPLOYING LOGIC Rawls’ Theory of Social Justice All social goods should be distributed equally, unless unequal distribution is to the advantage of the least favored Share the server evenly among all jobs Processor Sharing (PS) is fair

EMPLOYING LOGIC Min-Max fairness (Pareto Efficiency) All jobs deserve an equal share of the resources ... but if some jobs can use more without hurting others, that’s okay If you beat PS for all sizes, you’re still fair

? ? MIN-MAX FAIRNESS Definition: Consider an M/GI/1 queue. Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if: Is 1/(1-ρ) really a min-max criteria ? Is dividing by “x” the right scaling? ?

? ? Is 1/(1-ρ) Is dividing really a by “x” the right min-max criteria scaling? ? Theorem: In an M/GI/1 with an unbounded, continuous service distribution having finite E[X2], under any non-idling policy we have and further [Wierman and Harchol-Balter 2003]

? MIN-MAX FAIRNESS Definition: Consider an M/GI/1 queue. Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if: How do common policies perform? ?

PS – FCFS – SRPT Processor Sharing E[T(x)] / x PS x

First Come First Served PS – FCFS – SRPT First Come First Served The unfairness can be unbounded E[T(x)] / x PS FCFS x Under a Pareto with ρ=0.8, this is >80% of the jobs

Shortest Remaining Processing Time PS – FCFS – SRPT Shortest Remaining Processing Time x E[T(x)] / x x E[T(x)] / x SRPT PS PS SRPT Theorem SRPT is fair for ρ<0.5, regardless of the job size distribution. Theorem For every service distribution with finite E[X2], SRPT is unfair for high enough load.

OTHER POLICIES Always Unfair SRPT PS Always Fair Sometimes Fair FCFS

OTHER POLICIES Always Unfair Always Fair Sometimes Fair Prioritize PS PLCFS SRPT LRPT Prioritize Small Jobs Prioritize Large Jobs Always Fair PSJF PLJF LCFS Sometimes Fair ROS FCFS SJF LJF FB [Wierman and Harchol-Balter 2003, 2005]

! OTHER POLICIES Always Unfair Always Fair Sometimes Fair Remaining size based Always Unfair PS PLCFS SRPT LRPT Preemptive size based Non-preemptive non-size based Always Fair PSJF PLJF LCFS Sometimes Fair ROS FCFS There’s a tradeoff between fairness and efficiency ! SJF LJF Age based Non-preemptive size based FB [Wierman and Harchol-Balter 2003, 2005]

FAIRNESS VS. EFFICIENCY Always Unfair PS PLCFS SRPT LRPT Always Fair PSJF PLJF LCFS Sometimes Fair ROS Is there a fair policy with near optimal performance? ? FCFS SJF LJF more circles better mean response time FB

Fair Sojourn Protocol (FSP) [Henderson and Friedman 2004] Guarantees that every job finishes earlier than it would under PS.

Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times” Compute the time when each job would finish in a virtual PS system. Order the jobs according to their virtual completion times. Devote the full processor to the job with the earliest PS completion time

Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times” FSP did the same thing as SRPT !

Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times” How good are response times under FSP? ?

How good are response times under FSP? ? ...your answer here...

! MIN-MAX FAIRNESS Definition: Consider an M/GI/1 queue. Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if: Worries about fairness are not limited to the mean !

Central moments Var[T(x)], etc Raw moments E[T(x)i] Central moments Var[T(x)], etc Cumulant moments E[T(x)]  ? X BEYOND EXPECTATION: Higher Moments X [Wierman and Harchol-Balter 2005]

? CUMULANTS Do these look familiar? Cumulants are a descriptive statistic, similar to the moments. They can be found as a function of the moments: or from the log of the moment generating function: CUMULANTS Do these look familiar? ?

WHY CUMULANTS? Cumulants have many nice properties: additivity: homogeneity: 1st cumulant is shift-equivariant & the rest are shift-invariant

! ? MIN-MAX FAIRNESS Definition: Consider an M/GI/1 queue. Wierman and Harchol-Balter 2005 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if, for all i: Why is this the right generalization? ? Lots of open questions here !

MIN-MAX FAIRNESS IS NOT ALWAYS APPROPRIATE FCFS is the most fair policy in some settings

TEMPORAL FAIRNESS It is unfair to violate the seniority of a job Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. [Wierman 2004]

First Come First Served Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. FCFS – PLCFS – PS – SRPT First Come First Served x E[Pol(x)] FCFS 1 1-ρ

Preemptive Last Come First Served Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. FCFS – PLCFS – PS – SRPT Preemptive Last Come First Served x E[Pol(x)] FCFS 1 1-ρ PLCFS

FCFS – PLCFS – PS – SRPT Processor Sharing 1 1-ρ E[Pol(x)] PS PS PLCFS PLCFS x job size %-tile 1

Shortest Remaining Processing Time FCFS – PLCFS – PS – SRPT Shortest Remaining Processing Time FCFS FCFS 1 1-ρ SRPT E[Pol(x)] SRPT PLCFS PLCFS Prioritizing small jobs helps overall politeness ! x job size %-tile 1

MANY INTERESTING QUESTIONS... Can you think of a policy (other than FCFS) that avoids being maximally impolite to some job size? How polite are scheduling heuristics & techniques? What is the tradeoff between temporal fairness and proportional fairness? and efficiency?

PROPORTIONAL FAIRNESS VS TEMPORAL FAIRNESS VS EFFICIENCY min-max fairness politeness mean response time

Min-max Fairness Politeness less fair more fair less polite more polite Theorem In an M/GI/1 any Always Fair policy has FCFS SRPT FCFS FSP PS PLCFS FSP SRPT PS LRPT LRPT PLCFS

Min-max Fairness Politeness less fair more fair FCFS less polite more polite FCFS SRPT FSP PS more circles better mean response time LRPT PLCFS

WRAPUP

FAIRNESS IS AN AMORPHOUS CONCEPT a universal definition is unlikely to be useful (or possible) PROPORTIONAL FAIRNESS min-max TEMPORAL FAIRNESS politeness Prioritizing small jobs provides a balance between fairness & efficiency ! ...THIS IS A VERY ACTIVE AREA OF RESEARCH, MANY OPEN QUESTIONS REMAIN

AN EXPLOSION OF WORK ON FAIRNESS since the introduction of min-max fairness in 2003 defining other fairness metrics analyzing policies [Wierman 2004] [Raz, Avi-Itzhak 2004] [Levy, Raz, Avi-Itzhak 04] [Sandmann 2005] Many papers by: Wierman, Harchol-Balter, Henderson, Friedman, Biersack, Rai, Ayesta, Aalto, Nunez-Queija, Misra, Feng, Vernon, Williamson, Brown, Bansal, and others extending definition to higher moments [Wierman & Harchol-Balter 2005a,b] [Williamson, Gong 2003, 2004] [Brown 2006] and others.

MANY OTHER INTERESTING METRICS percent of service given to job i fair service percentage [Levy, Raz, Avi-Itzhak 04] DiscFreq = ni + c∙mi ni = number of jobs that arrived later and completed earlier than job i mi = number of larger jobs (at the arrival of job i) that complete earlier than job i [Sandmann 2005]

Adam Wierman Carnegie Mellon University acw@cs. cmu Adam Wierman Carnegie Mellon University acw@cs.cmu.edu Papers are available at: http://www.cs.cmu.edu/~acw