Download presentation

Presentation is loading. Please wait.

Published byAmina Braswell Modified over 3 years ago

1
VARUN GUPTA Carnegie Mellon University 1 With: Mor Harchol-Balter (CMU)

2
2 K homogeneous First-Come-First-Served servers Exponentially-distributed job sizes Load Balancer Knows queue lengths Not job sizes Q: What is the optimal load balancing policy? A: Join-the-Shortest-Queue Q: Why? A: JSQ = Minimize Expected Response time of arrival GOAL: Minimize Mean Response Time E[T] Poisson( λ )

3
3 μ=4μ=4 μ=4μ=4 μ =1 K heterogeneous First-Come-First-Served servers Exponentially-distributed job sizes Load Balancer Knows queue lengths Not job sizes Q: What is the optimal load balancing policy? μ =1 GOAL: Minimize Mean Response Time E[T] Poisson( λ )

4
Smart-JSQ = Join- Shortest-Queue (with smart tie breaks) 4 MER = Minimum Expected Response time μ=4μ=4 μ =1 μ=4μ=4 μ=4μ=4 μ=4μ=4 Q: Which is the better policy? Q: What is the optimal policy?

5
5 Outline Many-servers limit: Simulation Results Effect of K Effect of arrival rate ( λ ) Effect of degree of heterogeneity Light-traffic regimeHeavy-traffic regime Partial characterization of the optimal policy Complete characterization of optimal policies First asymptotic approximations

6
6 4 4 1 Poisson( λ ) 1 K 1 = α 1 K K 2 = α 2 K

7
7 4 4 1 K 1 = K/3 K 2 = 2K/3 Poisson( λ ) Q: Performance of MER 1 Case 1: λ < 4K/3 Fast can handle λ Arrivals find at least one fast idle E[T] = 1/4 Case 2: λ > 4K/3 Fast can not handle λ Can not use slow until each fast has 3 jobs !

8
8 4 4 1 K 1 = K/3 K 2 = 2K/3 Poisson( λ ) Q: Performance of Smart-JSQ 1 Case 1: λ < 4K/3 Fast can handle λ Arrivals find at least one fast idle E[T] = 1/4 Case 2: λ > 4K/3 Use slow as soon as each fast has 1 job !

9
9 4 4 1 K 1 = K/3 K 2 = 2K/3 Poisson( λ ) Smart-JSQ better than MER! …but any policy which sends to slow when all fast are busy is identical in light-traffic 1

10
HYBRID (smart- JSQ+MER) Smart-JSQ 10 MER μ=4μ=4 μ =1 μ =4 μ =1 Light-traffic HYBRID = Smart-JSQ μ=4μ=4 μ =1 μ =4 μ =1 μ=4μ=4 μ=4μ=4 smart-JSQ when some server idle MER when all busy

11
11 Outline Many-servers limit: Simulation Results Effect of K Effect of arrival rate ( λ ) Effect of degree of heterogeneity Light-traffic regimeHeavy-traffic regime Partial characterization of the optimal policy Complete characterization of optimal policies First asymptotic approximations

12
12 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ ) GOAL Analysis of policies for heterogeneous servers Analysis of JSQ for homogeneous server

13
13 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ ) GOAL Analysis of policies for heterogeneous servers Analysis of JSQ for homogeneous server

14
14 K Poisson( λ ) Analysis technique: Markov chain for total jobs in system 01K2K+22K+12K2K3K/2 λ λλλλ λ ? = mean departure rate given 3K/2 jobs

15
15 N = 3K/2 Poisson( λ ) K/2 Rate = K/2 Rate = K Departure rate = K–1 (not K) Finding the O(1) fluctuations critical to analysis O(1) idle queues

16
16 N = (1+ γ )K (0 <γ< 1) Poisson( λ ) γKγK (1- γ )K Rate = (1- γ )K Rate = K Departure rate = K–(1- γ )/ γ (not K) Finding the O(1) fluctuations critical to analysis O(1) idle queues

17
17 K Poisson( λ ) Analysis technique: Markov chain for total jobs in system 01K2K+22K+12K2K3K/2 λ λλλλ λ KK K-1 Asymptotically negligible probability mass First closed-form approx for JSQ!

18
18 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ ) GOAL Analysis of policies for heterogeneous servers Analysis of JSQ for homogeneous server

19
19 OPT policy maximize departure rate for each N (preemptively) send jobs to slow servers even when they have 1 job and all fast servers have > 1 Smart-JSQ is optimal in many-servers 4 4 1 1 K 1 = α 1 K K 2 = α 2 K Poisson( λ )

20
20 Outline Many-servers limit: Simulation Results Effect of K Effect of arrival rate ( λ ) Effect of degree of heterogeneity Light-traffic regimeHeavy-traffic regime Partial characterization of the optimal policy Complete characterization of optimal policies First asymptotic approximations

21
21 Smart-JSQ

22
22 MER Smart-JSQ

23
23 MER Smart-JSQ HYBRID

24
24 MER Smart-JSQ HYBRID

25
25 Smart-JSQ

26
26 MER Smart-JSQ

27
27 MER Smart-JSQ HYBRID

28
A new many-servers heavy-traffic scaling to analyze load balancing policies First closed-form approx of load balancing heuristics Choosing the right load balancer Few servers, Small load, High heterogeneity HYBRID Many servers, High load, Low heterogeneity smart-JSQ 28

29
29 MER Smart-JSQ HYBRID

30
30 MER Smart-JSQ HYBRID

31
31 MER Smart-JSQ HYBRID

Similar presentations

Presentation is loading. Please wait....

OK

Year 6 mental test 10 second questions

Year 6 mental test 10 second questions

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on network switching Ppt on life of albert einstein Ppt on recycling of waste materials Ppt online compressor sales Ppt on arithmetic micro operations Ppt on business model canvas Ppt on computer viruses and worms Download ppt on fdi in retail in india Ppt on nitrogen cycle and nitrogen fixation is a process Ppt on team building stories