Download presentation

Presentation is loading. Please wait.

Published byAmina Braswell Modified over 3 years ago

1
VARUN GUPTA Carnegie Mellon University 1 With: Mor Harchol-Balter (CMU)

2
2 K homogeneous First-Come-First-Served servers Exponentially-distributed job sizes Load Balancer Knows queue lengths Not job sizes Q: What is the optimal load balancing policy? A: Join-the-Shortest-Queue Q: Why? A: JSQ = Minimize Expected Response time of arrival GOAL: Minimize Mean Response Time E[T] Poisson( λ )

3
3 μ=4μ=4 μ=4μ=4 μ =1 K heterogeneous First-Come-First-Served servers Exponentially-distributed job sizes Load Balancer Knows queue lengths Not job sizes Q: What is the optimal load balancing policy? μ =1 GOAL: Minimize Mean Response Time E[T] Poisson( λ )

4
Smart-JSQ = Join- Shortest-Queue (with smart tie breaks) 4 MER = Minimum Expected Response time μ=4μ=4 μ =1 μ=4μ=4 μ=4μ=4 μ=4μ=4 Q: Which is the better policy? Q: What is the optimal policy?

5
5 Outline Many-servers limit: Simulation Results Effect of K Effect of arrival rate ( λ ) Effect of degree of heterogeneity Light-traffic regimeHeavy-traffic regime Partial characterization of the optimal policy Complete characterization of optimal policies First asymptotic approximations

6
6 4 4 1 Poisson( λ ) 1 K 1 = α 1 K K 2 = α 2 K

7
7 4 4 1 K 1 = K/3 K 2 = 2K/3 Poisson( λ ) Q: Performance of MER 1 Case 1: λ < 4K/3 Fast can handle λ Arrivals find at least one fast idle E[T] = 1/4 Case 2: λ > 4K/3 Fast can not handle λ Can not use slow until each fast has 3 jobs !

8
8 4 4 1 K 1 = K/3 K 2 = 2K/3 Poisson( λ ) Q: Performance of Smart-JSQ 1 Case 1: λ < 4K/3 Fast can handle λ Arrivals find at least one fast idle E[T] = 1/4 Case 2: λ > 4K/3 Use slow as soon as each fast has 1 job !

9
9 4 4 1 K 1 = K/3 K 2 = 2K/3 Poisson( λ ) Smart-JSQ better than MER! …but any policy which sends to slow when all fast are busy is identical in light-traffic 1

10
HYBRID (smart- JSQ+MER) Smart-JSQ 10 MER μ=4μ=4 μ =1 μ =4 μ =1 Light-traffic HYBRID = Smart-JSQ μ=4μ=4 μ =1 μ =4 μ =1 μ=4μ=4 μ=4μ=4 smart-JSQ when some server idle MER when all busy

11
11 Outline Many-servers limit: Simulation Results Effect of K Effect of arrival rate ( λ ) Effect of degree of heterogeneity Light-traffic regimeHeavy-traffic regime Partial characterization of the optimal policy Complete characterization of optimal policies First asymptotic approximations

12
12 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ ) GOAL Analysis of policies for heterogeneous servers Analysis of JSQ for homogeneous server

13
13 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ ) GOAL Analysis of policies for heterogeneous servers Analysis of JSQ for homogeneous server

14
14 K Poisson( λ ) Analysis technique: Markov chain for total jobs in system 01K2K+22K+12K2K3K/2 λ λλλλ λ ? = mean departure rate given 3K/2 jobs

15
15 N = 3K/2 Poisson( λ ) K/2 Rate = K/2 Rate = K Departure rate = K–1 (not K) Finding the O(1) fluctuations critical to analysis O(1) idle queues

16
16 N = (1+ γ )K (0 <γ< 1) Poisson( λ ) γKγK (1- γ )K Rate = (1- γ )K Rate = K Departure rate = K–(1- γ )/ γ (not K) Finding the O(1) fluctuations critical to analysis O(1) idle queues

17
17 K Poisson( λ ) Analysis technique: Markov chain for total jobs in system 01K2K+22K+12K2K3K/2 λ λλλλ λ KK K-1 Asymptotically negligible probability mass First closed-form approx for JSQ!

18
18 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ ) GOAL Analysis of policies for heterogeneous servers Analysis of JSQ for homogeneous server

19
19 OPT policy maximize departure rate for each N (preemptively) send jobs to slow servers even when they have 1 job and all fast servers have > 1 Smart-JSQ is optimal in many-servers 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ )

20
20 Outline Many-servers limit: Simulation Results Effect of K Effect of arrival rate ( λ ) Effect of degree of heterogeneity Light-traffic regimeHeavy-traffic regime Partial characterization of the optimal policy Complete characterization of optimal policies First asymptotic approximations

21
21 Smart-JSQ

22
22 MER Smart-JSQ

23
23 MER Smart-JSQ HYBRID

24
24 MER Smart-JSQ HYBRID

25
25 Smart-JSQ

26
26 MER Smart-JSQ

27
27 MER Smart-JSQ HYBRID

28
A new many-servers heavy-traffic scaling to analyze load balancing policies First closed-form approx of load balancing heuristics Choosing the right load balancer Few servers, Small load, High heterogeneity HYBRID Many servers, High load, Low heterogeneity smart-JSQ 28

29
29 MER Smart-JSQ HYBRID

30
30 MER Smart-JSQ HYBRID

31
31 MER Smart-JSQ HYBRID

Similar presentations

Presentation is loading. Please wait....

OK

Understanding Generalist Practice, 5e, Kirst-Ashman/Hull

Understanding Generalist Practice, 5e, Kirst-Ashman/Hull

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google