Queueing Theory

Queueing Theory Specification of a Queue Source Arrival Process Finite Infinite Arrival Process Service Time Distribution Maximum Queueing System Capacity Number of Servers Queue Discipline 30 1

Characteristics of Queuing Systems Key elements of queuing systems • Customer:-- refers to anything that arrives at a facility and requires service, e.g., people, machines, trucks, emails. • Server:-- refers to any resource that provides the requested service, eg. repairpersons, retrieval machines, runways at airport.

Queuing examples System Customers Server Reception desk People Receptionist Hospital Patients Nurses Airport Airplanes Runway Road network Cars Traffic light Grocery Shoppers Checkout station Opencast Mining Dumper Shovel

Components of a Queuing System Arrival Process Servers Queue or Waiting Line Service Process Exit

Parts of a Waiting Line Shovel Population of dumpers Arrivals from the general population … Queue (waiting line) Service facility Exit the system Population of dumpers Shovel enter exit Arrivals to the system In the system Exit the system Waiting Line Characteristics Limited vs. unlimited Queue discipline Service Characteristics Service design Statistical distribution of service Arrival Characteristics Size of the population Behavior of arrivals Statistical distribution of arrivals

1. Arrival Process 2. Queue Structure According to source According to numbers According to time 2. Queue Structure First-come-first-served (FCFS) Last-come-first-serve (LCFS) Service-in-random-order (SIRO) Priority service

Important queuing models with FIFO discipline The M/M/1 model The M/M/c model The M/M/c/K model (limited queuing capacity) The M/M/c//N model (limited calling population)

Kendal Notation Six parameters in shorthand First three typically used, unless specified Arrival Distribution Probability of a new dumper arrives in time t Service Distribution Probability distribution dumper is serviced in time t Number of servers Total Capacity (infinite if not specified) Population Size (infinite) Service Discipline (FCFS/FIFO)

Distributions M: Exponential D: Deterministic (e.g. fixed constant) Ek: Erlang with parameter k Hk: Hyperexponential with parameter k G: General (anything) M/M/1 is the simplest ‘realistic’ queue

Kendal Notation Examples M/M/1: Exponential arrivals and service, 1 server, infinite capacity and population, FCFS (FIFO) M/M/m Same, but M servers G/G/3/20/1500/SPF General arrival and service distributions, 3 servers, 17 queue slots (20-3), 1500 total jobs, Shortest Packet First

3. Service system 1. A single service system. Queue Arrivals Service facility Departures after service Arrivals

2. Multiple, parallel server, single queue model Service facility Channel 1 Channel 2 Channel 3 Departures after service Queue Arrivals

3. Multiple, parallel facilities with multiple queues Model Customers leave Service station Queues Arrivals

4. Service facilities in a series Service station 1 Service station 2 Arrivals Phase 1 Phase 2 Queues Queues Customers leave

Queuing Models Deterministic queuing model :-- Probabilistic queuing model Deterministic queuing model :--  = Mean number of arrivals per time period µ = Mean number of units served per time period

Assumptions The source population has infinite size. The inter-arrival time has an exponential probability distribution with a mean arrival rate of  customer arrivals per unit time. There is no unusual customer behaviour. The service discipline is FIFO. The service time has an exponential probability distribution with a mean service rate of  service completions per unit time. The mean arrival rate is less than the mean service rate, i.e.,  < . There is no unusual server behaviour.

Service Utilization Factor Consider an M/M/1 queue with arrival rate =  and service intensity =   = Expected capacity demand per time unit  = Expected capacity per time unit  Similarly if there are c servers in parallel, i.e., an M/M/c system but the expected capacity per time unit is then c* 

Useful probability distributions Poisson distribution Exponential distribution

Single Channel Model  = Mean number of arrivals per time period µ = Mean number of units served per time period Ls = Average number of units (customers) in the system (waiting and being served) = Ws = Average time a unit spends in the system (waiting time plus service time)  µ –  1

Lq = Average number of units waiting in the queue Wq = Average time a unit spends waiting in the queue p = Utilization factor for the system 2 µ(µ – )  µ

P0 = Probability of 0 units in the system (that is, the service unit is idle) = 1 – Pn > k = Probability of more than k units in the system, where n is the number of units in the system =  µ k + 1

Birth-And-Death Process l N+1 2 1 N N-1 N-2 m .... State: In the long run, we have: Rate IN = Rate Out Principle 58 4

Birth-And-Death Process(cont.) Equation Expressing This: State Rate In = Rate Out 0 m1P1 = l0P0 1 l0P0 + m2P2 = (l1 + m1) P1 2 l1P1 + m3P3 = (l2 + m2) P2 .... ................... N-1 lN-2PN-2 + mNPN = (lN-1 + mN-1) PN-1 N lN-1PN-1 + mN+1PN+1 = (lN + mN) PN 30 5

Birth-And-Death Process(cont.) Finding Steady State Process: State 0: P1 = (l0 / m1) P0 1: P2 = (l1 / m2) P1 + (m1P1 - l0P0) / m2 = (l1 / m2) P1 + (m1P1 - m1P1) / m2 = (l1 / m2) P1 = 30 6

Birth-And-Death Process(cont.) Finding Steady State Process(cont.): State n-1: Pn = (ln-1 / mn) Pn-1 + (mn-1Pn-1- ln-2Pn-2) / mn = (ln-1 / mn) Pn-1 + (mn-1Pn-1- mn-1Pn-1) / mn = (ln-1 / mn) Pn-1 30 7

Birth-And-Death Process(cont.) Finding Steady State Process(cont.): N: Pn+1 = (ln / mn+1) Pn+ (mnPn - ln-1Pn-1) / mn+1 = (ln / mn+1) Pn To Simplify: Let C = (ln-1 ln-2 .... l0) / (mn mn-1 ......... m1) Then Pn = Cn P0 , N = 1, 2, .... 30 8

M/M/1 Recall: r = l / m < 1 (for steady-state) P0 (r0 = 1) Recall: r = l / m < 1 (for steady-state) Cn = (l / m)n = rn , for n = 1, 2, ... Pn = Cn · P0 The requirement that 30 9

M/M/1(cont.) 1) 2) , for any x, , if |x| < 1. Thus, Pn = (1 - r) rn , for n = 0, 1, 2,... Note: 30 10

M/M/1(cont.) Consequently, 30 11

M/M/1(cont.) Similarly, 30 12

Example In an opencast mine loading and transporting has been carried out by shovel dumper combination. There is only one shovel is in operation. Dumpers arrive at the rate of 10 dumpers per hour and it takes shovel on an average 5 minutes to load a dumper. Assume arrival is Poisson and service time is exponentially distributed. Find the proportion of time the shovel is busy, probability that the shovel is sitting idle, expected number of dumpers in the system, expected waiting time of the dumper in the queue. A queue of trucks at a crusher plant hopper is known to be M/M/I queue. The probability that there is no truck to unload is 0.3. Due to rains the mean service time at the hopper is increased by 30%. As a consequence, the expected number of trucks in the queuing system (including the one possibly unloading) becomes

M/M/s (s > 1) l s m 2m (s-1) ... 3m Recall: = s+1 2 1 s-1 s-2 m 2m (s-1) ... 3 3m Recall: n = , for n = 0, 1, 2,..... = n , for n = 1, 2,..., s = s , for n = s, s+1,... Rate Diagram 58 24

M/M/s (cont.) State Rate In = Rate Out 0 mP1 = lP0 1 2mP2 + lP0 = (l + m) P1 2 3mP3 + lP1 = (l + 2m) P2 .... ................... s-1 smPs + lPs-2 = {l + (s-1)m} Ps-1 s smPs+1 + lPs-1 = (l + sm) Ps s+1 smPs+2 + lPs = (l + sm) Ps+1 30 25

M/M/s (cont.) Now, solve for P1 , P2, P3... in terms of P0 P1 = (l / m) P0 P2 = (l / 2m) P1 = (1/2!) × (l / m)2 P0 P3 = (l / 3m) P2 = (1/3!) × (l / m)3 P0 ......... Ps = (1/s!) × (l / m)s P0 Ps+1 = (1/s) × (l / m) Ps = 30 26

M/M/s (cont.) 30 27

M/M/s (cont.) Therefore, if we denote Pn = Cn× P0 , then for n = 1, 2, ...., s. and , for n = s+1, s+2,... 30 28

M/M/s (cont.) if 0 £ n £ s if s £ n So, if l < sm => 30 29

M/M/s (cont.) Now solve for Lq: Note, r = l / sm ; Note, n = s + j 30

M/M/s (cont.) 30 31

M/M/s (cont.) , r = l / sm (Lq : avg # in queue) Wq = Lq / l (Wq: avg waiting time in Q) W = Wq + 1 / m (W: avg waiting time in sys.) L = l (Wq + 1/m) (L: avg # in the system) = Lq + l / m 30 32

Steady-State Parameters of M/M/s Queue r = l / sm P(L(¥) ³ s) = {(l/m)s P0} / {s!(1- l/sm)} = {(sr)s P0} / {s! (1 - r)} 30 16

Steady-State Parameters of M/M/s Queue (cont.) L = sr + {(sr)s+1 P0} / {s (s!) (1 - r)2} = sr + {r P (L(¥) ³ s) } / {1 - r} W = L / l Wq = W - 1/m Lq = l Wq = {(sr)s+1 P0} / {s (s!) (1 - r)2} = {r P (L(¥) ³ s) } / {1 - r} L - Lq = l / m = sr 30 16

M/M/s Case Example I ....... Example: M/M/2 ; s = 2 l = 1/ 10 , m 1/8 2(1/8) 2 1 1/10 ....... 3 Example: M/M/2 ; s = 2 l = 1/ 10 , m = 1/8 (=service rate/server) r = l / s m = {1/10} / {2(1/8) } = 0.4 30 33

M/M/s Case Example I (cont.) = 0.429 (@ 43% of time, system is empty) as compared to s = 1: P0 = 0.20 30 34

M/M/s Case Example I (cont.) Wq = Lq / l = 0.152 / (1/10) = 1.52 (min) W = Wq + 1 / m = 1.52 + 1 / (1/8) = 9.52 (min) What proportion of time is both repairman busy? (long run) P(N ³ 2) = 1 - P0 - P1 = 1 - 0.429 - 0.343 = 0.228 (Good or Bad?) 30 35

M/M/s Example II Many early examples of queueing theory applied to practical problems concerning tool cribs. Attendants manage the tool cribs while mechanics, assumed to be from an infinite calling population, arrive for service. Assume Poisson arrivals at rate 2 mechanics per minute and exponentially distributed service times with mean 40 seconds. 30 45

M/M/s Example II (cont.) l = 2 per minute, and m = 60/40 = 3/2 per minute. Since, the offered load is greater than 1, that is, since, l / m = 2 / (3/2) = 4/3 > 1, more than one server is needed if the system is to have a statistical equilibrium. The requirement for steady state is that s > l / m = 4/3. Thus, at least s = 2 attendants are needed. The quantity 4/3 is the expected number of busy server, and for s ³ 2, r = 4 / (3s) is the long-run proportion of time each server is busy. (What would happen if there were only s = 1 server?) 30 45

M/M/s Example II (cont.) Let there be s = 2 attendants. First, P0 is calculated as = {1 + 4/3 + (16/9)(1/2)(3)} -1 = {15 / 3}-1 = 1/5 = 0.2 The probability that all servers are busy is given by P(L(¥) ³ 2) = {(4/3)2 (1/5)} / {2!(1- 2/3)} = (8/3) (1/5) = 0.533 30 16

M/M/s Example II (cont.) Thus, the time-average length of the waiting line of mechanics is Lq = {(2/3)(8/15)} / (1 - 2/3) = 1.07 mechanics and the time-average number in system is given by L = Lq+ l/m = 16/15 + 4/3 = 12/5 = 2.4 mechanics Using Little’s relationships, the average time a mechanic spends at the tool crib is W = L / l = 2.4 / 2 = 1.2 minutes while the avg time spent waiting for an attendant is Wq = W - 1/m = 1.2 - 2/3 = 0.533 minute 30 16

M/M/1/N (single server) 1 2 N-2 N-1 N N+1 3   ... Rate Diagram Undefined 58 36

M/M/1/N (cont.) 1. Form Balance Equations: 2. Solve for P0: or P0 + (l/m)1 P0 +    + (l/m)N P0 = 1 P0 {1+ (l/m)1 +    + (l/m)N } = 1 P0 = 1 / { } = (1 - r) / (1 - rN+1) 30 37

M/M/1/N (cont.) So, , for n = 0, 1, 2, ..., N Hence, 30 38

M/M/1/N (cont.) 30 39

M/M/1/N (cont.) As usual (when s = 1) Lq = L - (1- P0) W = L / le , where le = l (1 - PN) Wq = Lq / le 30 40

M/M/1/N Example The unisex barbershop can hold only three customers, one in service and two waiting. Additional customers are turned away when the system is full. Determine the measures of effectiveness for this system. The traffic intensity is l / m = 2 / 3. The probability that there are three customers in the system is computed by Pn = P3 = {(1-2/3) (2/3)3} / { 1 - (2/3)4} = 8 / 65 = 0.123 30 41

M/M/1/N Example (cont.) The expected # of customers in the shop is given by Now, the effective arrival rate, le , is given by le = l (1 - Pn) = 2(1 - 8/65) = 2 × 57 / 65 =114/65 = 1.754 (customers/hour) Then W can be calculated as W = L / le = 1.015 / 1.754 = 0.579 (hour) 30 42

M/M/1/N Example (cont.) In order to calculate Lq, first determine P0 as P0 = (1 - r) / (1 - rN+1) = (1 - 2/3) / {1 - (2/3)4} = {1/3} / {65/81} = 27 / 65 = 0.415 Then the average length of the queue is given by Lq = L - (1- P0) = 1.015 - (1 - 0.415) = 0.43 (customer) 30 43

M/M/1/N Example (cont.) Note that 1- P0 = 0.585 is the average number of customers being served, or equivalently, the probability that the single server is busy. Thus the server utilization, or proportion of time the server is busy in the long run, is given by r = 1- P0 = le / m = 0.585 Finally, the waiting time in the queue is determined by Little’s equation as Wq = Lq / le = 0.43 / 1.754 = 0.245 (hour) 30 44

M/M/1/N Example (cont.) The reader should compare these results to those of the unisex barbershop before the capacity constraint was placed on the system. Specifically, in systems with limited capacity, the traffic intensity l / m can assume any positive value and no longer equals the server utilization r = le / m. Note that server utilization decreases from 67% to 58.5% when the system imposes a capacity constraint. 30 45

M/M/1/N Example (cont.) Since P0 and P3 have been computed, it is easy to check the value of L using equation To make the check requires computation of P1 & P2: P1 = {(1 - 2/3)(2/3)} / {1- (2/3)4} = 18/65 = 0.277 Since P0 + P1 + P2 + P3 = 1, P2 = 1 - P0 - P1 - P3 = 1 - 27/65 - 18/65 - 8/65 = 12 / 65 = 0.185 30 46

M/M/1/N Example (cont.) L = = 0×(27/65) + 1×(18/65) + 2×(12/65) + 3×(8/65) = 66 / 65 = 1.015 (customer) which is the same value as the expected number computed. 30 47

M/M/s/N l m 2m ... Undefined 2 1 Rate Diagram N+1 N N-1 s-1 s+1 s 58 N+1 2 1 N N-1 l m 2m ... Rate Diagram Undefined s-1 s+1 s 58 36

Steady-State Parameters of M/M/s/N for n = 1, 2, ... s for n = s, s+1, ... N 0, for n > N 30 16

Steady-State Parameters of M/M/s/N (cont.) Note: W and Wq are obtained from these quantities just as shown for the single server case. 30 16

Steady-State Parameters of M/G/1 Queue r = l / m L = r + {l2 (m-2 + s2)} / {2 (1 - r)} = r + {r2 (1 + s2 m2)} / {2 (1 - r)} W = m-1 + {l (m-2 + s2)} / {2 (1 - r)} Wq = {l (m-2 + s2)} / {2 (1 - r)} Lq = {l2 (m-2 + s2)} / {2 (1 - r)} = {r2 (1 + s2 m2)} / {2 (1 - r)} P0 = 1 - r 30 16

M/G/1 Example There are two workers competing for a job. Able claims an average service time which is faster than Baker’s, but Baker claims to be more consistent, if not as fast. The arrivals occur according to a Poisson process at a rate of l= 2 per hour. (1/30 per minute). Able’s statistics are an average service time of 24 minutes with a standard deviation of 20 minutes. Baker’s service statistics are an average service time of 25 minutes, but a standard deviation of only 2 minutes. If the average length of the queue is the criterion for hiring, which worker should be hired? 30 13

M/G/1 Example (cont.) For Able, l = 1/30 (per min), m-1 = 24 (min), r = l / m = 24/30 = 4/5 s2 = 202 = 400(min2) Lq = {l2 (m-2 + s2)} / {2 (1 - r)} = {(1/30)2 (242 + 400)} / {2 (1-4/5)} = 2.711 (customers) For Baker, l = 1/30 (per min), m-1 = 25 (min), r = l / m = 25/30 = 5/6 s2 = 22 = 4(min2) Lq = {(1/30)2 (252 + 4)} / {2 (1-5/6)} = 2.097 (customers) 30 16

M/G/1 Example (cont.) Although working faster on the average, Able’s greater service variability results in an average queue length about 30% greater than Baker’s. On the other hand, the proportion of arrivals who would find Able idle and thus experience no delay is P0 = 1 - r = 1 / 5 = 20%, while the proportion who would find Baker idle and thus experience no delay is P0 = 1 - r = 1 / 6 = 16.7%. On the basis of average queue length, Lq , Baker wins. 30 13

Steady-State Parameters of M/Ek/1 Queue 30 16

M/Ek/1 Example Patient arrive for a physical examination according to a Poisson process at the rate of one per hour. The physical examination requires three stages, each one independently and exponentially distributed with a service time of 15 minutes. A patient must go through all three stages before the next patient is admitted to the treatment facility. Determine the average number of delayed patients ,Lq , for this system. 30 13

M/Ek/1 Example (cont.) If patients follow this treatment pattern, the service-time distribution will be Erlang of order k=3. The necessary treatment parameters are l = 1/60 per minute and m = 1/45 per minute; thus 30 16

Steady-State Parameters of M/D/1 Queue 30 16

M/D/1 Example Arrivals to an airport are all directed to the same runway. At a certain time of the day, these arrivals are Poisson distributed at a rate of 30 per hour. The time to land an aircraft is a constant 90 seconds. Determine Lq, Wq, L and W for this airport. In this case l= 0.5 per minute, and 1/m = 1.5 minutes, or m = 2/3 per minute. 30 13

M/D/1 Example (cont.) The runway utilization is r = l / m = (1/2) / (2/3) = 3/4 The steady-state parameters are given by Lq = {(3/4) 2} / {2 (1 - 3/4)} = 9 / 8 = 1.125 aircraft Wq = Lq / l = (9/8) / (1/2) = 2.25 minutes W = Wq + 1 / m = 2.25 + 1.5 = 3.75 minutes L = Lq + l / m = 1.125 + 0.75 = 1.875 aircraft 30 13

Steady-State Parameters of M/G/¥ Queue P0 = e-l/m Pn = {e-l/m (l/m)n} / n! , n = 0, 1,... W = 1 / m Wq = 0 L = l / m Lq = 0 30 16

M/G/¥ Example Prior to introducing their new on-line computer information service, The Connection must plan their system capacity in terms of the number of users that can be logged on simultaneously. If the service is successful, customers are expected to log on at a rate of l = 500 per hour, according to a Poisson process, and stay connected for an average of 1/m = 20 minutes (or 1/3 hour). In the real system there will be an upper limit on simultaneous users, but for planning purpose The 30 45

M/G/¥ Example (cont.) Connection can pretend that the number of simultaneous users is infinite. An M/G/¥ model of the system implies that the expected number of simultaneous users is L = l/m = 500(3) = 1500, so a capacity greater than 1500 is certainly required. To ensure that they have adequate capacity 95% of the time, The Connection could allow the number of simultaneous users to be the smallest value s such that 30 45

M/G/¥ Example (cont.) A capacity of s=1564 simultaneous users satisfies this requirement. 30 45

Steady-State Parameters of M/M/s/K/K Queue n = 0, 1, ..., s-1 n = s, s+1, ... K 30 16

Steady-State Parameters of M/M/s/K/K Queue (cont.) W = L / le Wq = Lq / le r = (L - Lq) / s = le / sm 30 16

M/M/s/K/K Example There are two workers that are responsible for 10 milling machines. The machines run on the average of 20 minutes, then require an average 5-minute service period both times exponentially distributed. Therefore, l = 1/20 and m = 1/5. Determine the various measures of performance for this system. 30 45

M/M/s/K/K Example (cont.) All of the performance measures depend on P0 = 0.065 Using P0 we can obtain the other Pn, from which we can compute the average number of machines waiting for service 30 45

M/M/s/K/K Example (cont.) The effective arrival rate and the average waiting time in the queue Wq = Lq / le = 4.27 (minutes) Similarly, we can compute the expected number of machines being serviced or waiting to be served 30 45

M/M/s/K/K Example (cont.) The average number of machines being serviced is given by L - Lq = 3.17 - 1.46 = 1.71 (machines) since the machines must be running, waiting to be served, or in service, the average number of running machines is given by K - L = 10 - 3.17 = 6.83 (machines) A frequently asked question is: What will happen if the number of servers is increased or decreased? 30 45

M/M/s/K/K Example (cont.) If the number of workers in this example increases to three(s=3), then the time-average number of running machines increases to K - L = 7.74 (machines) an increase of 0.91 machine, on the average. Conversely, what happens if the number of servers decreases to one? Then the time-average number of running machines decreases to K - L = 3.98 (machines) 30 45

M/M/s/K/K Example (cont.) The decrease from two to one server has resulted in a drop of nearly three machines running, on the average. This example illustrates several general relationships that have been found to hold for almost all queues. If the number of servers is decreased, delays, server utilization, and the probability of an arrival having to wait to begin service all increase. 30 45