1. Professor Saad Haj Bakry, PhD, CEng, FIEE N ETWORK A RCHITECTURE I NTRODUCTION TO Q UEUEING S YSTEMS

2. C ONTENTS QUEUEING SYSTEMS Saad Haj Bakry B ASIC E LEMENTS S TEADY S TATE M / M / 1 M / M / 1 / x M / M / x 1 M / M / x / x M / M / infinity M / M / 1 network

3. B ASIC E LEMENTS QUEUEING SYSTEMS Saad Haj Bakry Q UEUEING AT AN I NSTANT S YSTEM P ERFORMANCE S YSTEM N OTATION

4. Departing Customers Blocked Customers 1 2 x1x1 Arriving Customers Waiting line (x 1 ) Servers Queuing System A1 Process A2 Process BASIC ELEMENTS QUEUEING SYSTEMS Saad Haj Bakry

5. CUSTOMERS ARE QUEUED IF ALL SERVERS ARE BUSY CUSTOMERS ARE TURNED AWAY IF ALL SERVERS ARE BUSY RANDOM CUSTOMER ARRIVALS BUSY VACANT LOSS SYSTEM DELAY SYSTEM BASIC ELEMENTS QUEUEING SYSTEMS Saad Haj Bakry SERVERS

6. S YSTEM N OTATION BASIC ELEMENTS QUEUEING SYSTEMS Saad Haj Bakry A 1 / A 2 / x 1 / x A 1 Arrival Process e.g. A 1 = M Poisson / Exponential Interarrival A 1 Arrival Process e.g. A 1 = M Poisson / Exponential Interarrival A 2 Service Process e.g. A 2 = M Exponential Service Time A 2 Service Process e.g. A 2 = M Exponential Service Time x 1 No. of Servers x 2 Queue Size x 1 No. of Servers x 2 Queue Size x Maximum no. of customers allowed in the system x = x 1 + x 2 x Maximum no. of customers allowed in the system x = x 1 + x 2

7. Q UEUEING AT AN I NSTANT BASIC ELEMENTS QUEUEING SYSTEMS Saad Haj Bakry N[t] No. of customers in the system N[t] <= x N[t]=Q[t]+S[t] N[t] No. of customers in the system N[t] <= x N[t]=Q[t]+S[t] Q[t] No. of customers waiting in the queue Q[t] <= x 2 Q[t] No. of customers waiting in the queue Q[t] <= x 2 S[t] No. of customers being served S[t] <= x 1 S[t] No. of customers being served S[t] <= x 1 t : Certain Instant

8. S YSTEM P ERFORMANCE BASIC ELEMENTS QUEUEING SYSTEMS Saad Haj Bakry L OSS R: rate of arrivals K: rate of service L: rate of loss. R = K + L Congestion = L / R L OSS R: rate of arrivals K: rate of service L: rate of loss. R = K + L Congestion = L / R D ELAY for customer [ i] T[i]: total delay V[i]: service Time W[i]: queuing delay T[i] = V[i] + W[i] D ELAY for customer [ i] T[i]: total delay V[i]: service Time W[i]: queuing delay T[i] = V[i] + W[i]

9. S TEADY S TATE QUEUEING SYSTEMS Saad Haj Bakry L ITTLE ’ S F ORMULA S ERVERS U TILIZATION

10. L ITTLE ’ S F ORMULA QUEUEING SYSTEMS Saad Haj Bakry S TEADY S TATE F ORMULA: Steady State N = R. T [Remember Newton] F ORMULA: Steady State N = R. T [Remember Newton] P ARAMETERS / A VERAGES N: Average of N[t] (expected number of customers in the system) R: arrival rate [customer / time unit] T: average of T[i] (expected time spent by a customer in the system) P ARAMETERS / A VERAGES N: Average of N[t] (expected number of customers in the system) R: arrival rate [customer / time unit] T: average of T[i] (expected time spent by a customer in the system) C ONCLUSIONS Q = R. W S = R. V C ONCLUSIONS Q = R. W S = R. V

11. U TILIZATION QUEUEING SYSTEMS Saad Haj Bakry S TEADY S TATE S INGLE S ERVER Free: P[0] Busy: S = 1 - P[0] = R. V Utilization: U = S = R. V S INGLE S ERVER Free: P[0] Busy: S = 1 - P[0] = R. V Utilization: U = S = R. V ( x 1 ) S ERVERS: U = S / x 1 U = (R.V) / x 1

12. M / M / 1 Q UEUE QUEUEING SYSTEMS Saad Haj Bakry P OISSON A RRIVALS E XPONENTIAL D ISTRIBUTION M ARKOV C HAIN C HANGE P ROPERTIES S TEADY S TATE D ELAY M / G(D) / 1 A PPLICATIONS

13. P OISSON A RRIVALS QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 P ARAMETERS R: rate of arrivals t : time interval (R.t): average arrivals z : possible arrivals P[z]: probability of [z] P ARAMETERS R: rate of arrivals t : time interval (R.t): average arrivals z : possible arrivals P[z]: probability of [z] E QUATION P(z) = (R z /z!).e (-R) for: t=1 Variance: Rt : R Illustration: Draw (z,P[z]) for R

14. E XPONENTIAL D ISTRIBUTION QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 P ARAMETERS R: rate of arrivals. d: period between consecutive arrivals D: average period (d) D = (1/R) P ARAMETERS R: rate of arrivals. d: period between consecutive arrivals D: average period (d) D = (1/R) E QUATION P[d] = R.e (-Rd) Variance: (1/R 2 ) Illustration: Draw (d,P[d]) for (1/R) E QUATION P[d] = R.e (-Rd) Variance: (1/R 2 ) Illustration: Draw (d,P[d]) for (1/R)

15. M ARKOV C HAIN QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 A RRIVAL P ROCESS: Poisson Arrival (Rate: R [arrivals / time unit]) Exponential Inter-arrival Time: “ iid: independently identically distributed ” (Average: (1/R) [time units]) Memory-less Process A RRIVAL P ROCESS: Poisson Arrival (Rate: R [arrivals / time unit]) Exponential Inter-arrival Time: “ iid: independently identically distributed ” (Average: (1/R) [time units]) Memory-less Process S ERVICE P ROCESS: Exponential Service Time (Average: V [time units]): ” iid ” N[t] : Number of Customers in the System at Time (t) in an M/M/1 system is a “ Continuous Time Markov Chain ”

16. N(t) C HANGE P ROPERTIES QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 A SSUMPTION: No 2 Events or more during “ dt ” : 2 arrivals 2 services Mixed A SSUMPTION: No 2 Events or more during “ dt ” : 2 arrivals 2 services Mixed 1 A RRIVAL D URING “ dt ” : P [1] = R. dt 1 S ERVICE D URING “ dt ” : p [service time = dt] = (1/V). dt

17. N: A VERAGE N UMBER OF C US. IN THE S YS.: S TEADY S TATE A NALYSIS QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 S TEADY S TATE: P[1]. (1/V) dt = P[0]. R. dt P[2]. (1/V) dt = P[1]. R. dt R ESULT: P[j] = (RV) j. P[0] = U j (1-U) F ACTS: F ACTS: P[j] = P(0)[R.V] j = P[0].(1/(1-R.V)) = 1 P[0] = (1-U) N =j. P[j]= P[0] j. (R.V) j

18. S TEADY S TAT E A NALYSIS: R EMEMBER QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 E XPONENTIAL E QUATION: e x = 1 + x + (x 2 /2!) + (x 3 /3!) + (x 4 /4!) +.. to infinity G EOMETRIC E QUATION: S = a + a.r + a.r 2 + a.r 3 +..+ ar (n-1) = a.(1-r n ) / (1-r) [ For (r < 1) and (n very large)] S = a / (1-r) S = r + 2r 2 + 3r 3 + 4r 4 +...+ir i +...to infinity= ( r / (1-r) 2 ) [for a < 1]

19. S YSTEM D ELAY QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 N = (1-U) (j.U j ) = (1-U).(U/(1-U) 2 ) = U / (1-U) N = R. T T = N / R Q = R. W W = T - V N = R. T T = N / R Q = R. W W = T - V T = V / (1 - U) Q = U 2 / (1 - U) W = (U. V) / (1 - U) Condition: U < 1 T = V / (1 - U) Q = U 2 / (1 - U) W = (U. V) / (1 - U) Condition: U < 1 No Blocking: All Items Served

20. M / G / 1 & M / D / 1 QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 M / G / 1: General Independent Service Time N = U + ( (U 2. A) / (1-U) ) T = V + ( (U.V.A) / (1-U) ) Q = (U 2. A) / (1-U) W = (U.V.A) / (1-U) A = (1/2) ( 1+ (Y V /V) 2 ) Y V : Standard Deviation for Service Time (v) M / G / 1: General Independent Service Time N = U + ( (U 2. A) / (1-U) ) T = V + ( (U.V.A) / (1-U) ) Q = (U 2. A) / (1-U) W = (U.V.A) / (1-U) A = (1/2) ( 1+ (Y V /V) 2 ) Y V : Standard Deviation for Service Time (v) M / D / 1: Deterministic: Fixed Length Service Time N = U + ( U 2 / (2.(1-U)) ) T = ( V.(2-U) ) / ( 2(1-U) ) Q = U 2 / ( 2(1-U) ) W = (U.V) / ( 2(1-U) ) M / D / 1: Deterministic: Fixed Length Service Time N = U + ( U 2 / (2.(1-U)) ) T = ( V.(2-U) ) / ( 2(1-U) ) Q = U 2 / ( 2(1-U) ) W = (U.V) / ( 2(1-U) ) “ G ” = “ M ” : A = 1 “ G ” = “ D ” : A = 1/2

21. A PPLICATION: C ONCENTRATOR QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 G IVEN: R = 1 (arr. / 4 ms) V = 3 (ms) F IND: N & T R increase for 100% in T R = 0.25 (arr. / ms) = (6/24) U = R.V = 0.75 N = U / (1-U) = 3 (in the system) T = N / R = 12 (ms) T new = 24 (ms) = V / (1 - R new.V) R new = ( T new - V) / (V. T new ) = (7/24) R [increase] = (R new - R old ) / R old = (1/6) = 17 %

22. A PPLICATION: P ROCESSORS QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 G IVEN: K Processors with r &v versus 1 Processor with R = K. r V = (v/K) G IVEN: K Processors with r &v versus 1 Processor with R = K. r V = (v/K) K Processors with r &v: t = v / (1-rv) = F IND: T “ versus ” t 1 Processors with R & V: T = V / (1-R.V) = (v/K) / (1 - ((K.r). (v/K)) T = (1/K). t

23. M / M / 1 / x Q UEUE QUEUEING SYSTEMS Saad Haj Bakry P ERFORMANCE M EASURES S TEADY S TATE L OAD: O FFERED / C ARRIED A PPLICATIONS

24. S TEADY S TATE QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 / x A LL S TATES: P[j] = P[0] (RV) j = 1 A LL S TATES: P[j] = P[0] (RV) j = 1 S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(1/V).dt = P[1].R.dt P[x].(1/V).dt = P[x-1].R.dt P(j) = (RV) j.P(0) (for j = 0 to x) P[j] = (U j.(1-U)) / (1-U x+1 ) S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(1/V).dt = P[1].R.dt P[x].(1/V).dt = P[x-1].R.dt P(j) = (RV) j.P(0) (for j = 0 to x) P[j] = (U j.(1-U)) / (1-U x+1 )

25. P ERFORMANCE QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 / x C ONGESTION: P[x] = (U x.(1-U)) / (1-U x+1 ) N = (U / (1-U)) - ( ((x+1).U x+1 ) / (1-U x+1 ) ) A CCEPTED R ATE: K = R. ( 1 - P[x] ) A VERAGE D ELAY: T = N / K = N / (R.(1-P(x)))

26. L OAD QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 / x O FFERED L OAD: Rate: R [cus / sec] Service Time: V [sec] Load: (R.V) S ERVICED L OAD: Rate: K [cus / sec] Service Time: V [sec] Load: (K.V) B LOCKED L OAD: ( V.(R-K) ) Congestion = P[x] = ( V.(R-K) ) / (R.V)

27. A PPLICATION: C OMPARISON QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 P[x] [M/M/1] = U x. (1-U) M / M / 1 / x M / M / 1 / x P[x] [M/M/1/x] = (U x.(1-U)) / (1-U x+1 ) = P[x] [M/M/1]. (1 / (1-U x+1 )) = P[x] [M/M/1]. [ 1 + U x+1 + (U x+1 ) 2 +..] M / M / 1 / x P[x] [M/M/1/x] = (U x.(1-U)) / (1-U x+1 ) = P[x] [M/M/1]. (1 / (1-U x+1 )) = P[x] [M/M/1]. [ 1 + U x+1 + (U x+1 ) 2 +..] U SING G EOMETRIC E QUATION

28. A PPLICATION: L OAD E VALUATION QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 / x P ROBLEM: Investigate the effect of the value of “ x ” on “ serviced load ” and on “ delay ” As “ x ” increases both the “ serviced load ” and “ delay ” increase: Demonstrate / Balance

29. M / M / C Q UEUE QUEUEING SYSTEMS Saad Haj Bakry P ERFORMANCE M EASURES S TEADY S TATE A PPLICATIONS

30. S TEADY S TATE QUEUEING SYSTEMS Saad Haj Bakry M / M / C O FFERED L OAD: A = R. V S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(2/V).dt = P[1].R.dt P[C {or more} ].(C/V).dt = P[C {or more} - 1].R.dt P[j] = ( (RV) j / j! ). P[0] (for j =C ) S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(2/V).dt = P[1].R.dt P[C {or more} ].(C/V).dt = P[C {or more} - 1].R.dt P[j] = ( (RV) j / j! ). P[0] (for j =C ) U TILIZATION: U = (R.V) / C = (A/C) P[C] = (A C / C!). P[0] P[C+1] = U.P[C] P[C+2] = U 2.P[C]

31. P ERFORMANCE: 1/2 QUEUEING SYSTEMS Saad Haj Bakry M / M / C P[0] = 1 /( (A j /j!) + (A C /C!).(1/(1-U)) ) P[N>=C] = P[C]. U (j-C) Erlang: E C [C,A] = P[C] / (1-U) P[N>=C] = P[C]. U (j-C) Erlang: E C [C,A] = P[C] / (1-U) W AITING P ROBABILITY

32. P ERFORMANCE: 2/2 QUEUEING SYSTEMS Saad Haj Bakry M / M / C W AITING T IME W AITING C USTOMERS Q = P[C]. (j-C).U (j-C) = (U / (1-U)). E C [C,A] Q = P[C]. (j-C).U (j-C) = (U / (1-U)). E C [C,A] T = W + V & W = Q / R W = E C [C,A]. (V / (C.(1-U)) T = W + V & W = Q / R W = E C [C,A]. (V / (C.(1-U))

33. A PPLICATION: C OMPARISON QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 INPUT: R = 0.5, V=1 FIND: U, W, T OUTPUT: U = R.V = 0.5 W = (U.V) / (1-U) = 1 T = V / (1-U) = 2 M / M / 1 INPUT: R = 0.5, V=1 FIND: U, W, T OUTPUT: U = R.V = 0.5 W = (U.V) / (1-U) = 1 T = V / (1-U) = 2 M / M / 2 INPUT: R = 0.5, V= 2 FIND: U, W, T OUTPUT: U = (R.V) / 2 = 0.5 W = E C [C,A]. (V / (C.(1-U)) W = (2/3) 2 M / M / 2 INPUT: R = 0.5, V= 2 FIND: U, W, T OUTPUT: U = (R.V) / 2 = 0.5 W = E C [C,A]. (V / (C.(1-U)) W = (2/3) 2 M / M / C

34. A PPLICATION: P ROBLEM QUEUEING SYSTEMS Saad Haj Bakry P ROBLEM: INPUT: C = 4 [telephone circuits] R = 0.5 [call / min] V = 4 [min] FIND: LOAD : A UTILIZATION: U ERLANG-C M / M / C A NSWERS: A = 2 U = 0.5 E C [C,A] = 0.17 A NSWERS: A = 2 U = 0.5 E C [C,A] = 0.17

35. M / M / C / C Q UEUE QUEUEING SYSTEMS Saad Haj Bakry P ERFORMANCE M EASURES S TEADY S TATE A PPLICATIONS

36. S TEADY S TATE QUEUEING SYSTEMS Saad Haj Bakry M / M / C / C O FFERED L OAD: A = R. V S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(2/V).dt = P[1].R.dt P[C].(C/V).dt = P[C-1].R.dt P[j] = ( (RV) j / j! ). P[0] P[0] = 1 / (A j / j!) S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(2/V).dt = P[1].R.dt P[C].(C/V).dt = P[C-1].R.dt P[j] = ( (RV) j / j! ). P[0] P[0] = 1 / (A j / j!) P[C] = ( (A) C / C! ) / (A j / j!) = ERLANG-B: E B [C,A] P[C] = ( (A) C / C! ) / (A j / j!) = ERLANG-B: E B [C,A]

37. P ERFORMANCE QUEUEING SYSTEMS Saad Haj Bakry M / M / C / C E B [C,A] = ( (A) C / C! ) / (A j / j!) E B [C,A] = ( (A) C / C! ) / (A j / j!) K = R. (1 - E B [C,A]) N = K. V U = (K. V) / C = N / C

38. A PPLICATION QUEUEING SYSTEMS Saad Haj Bakry A NSWERS: A = 2 [Erlang] E B [C,A] = 0.095 K = 0.4525 N = 1.81 U = 0.4525 A NSWERS: A = 2 [Erlang] E B [C,A] = 0.095 K = 0.4525 N = 1.81 U = 0.4525 M / M / C / C P ROBLEM: INPUT: C = 4 [telephone circuits] R = 0.5 [call / min] V = 4 [min] FIND: Load : A ERLANG-B: E B [C,A] Service Rate: K Average Calls: N

39. M / M / infinity Q UEUE QUEUEING SYSTEMS Saad Haj Bakry A NALYSIS A PPLICATIONS

40. A NALYSIS QUEUEING SYSTEMS Saad Haj Bakry M / M / infinity O FFERED L OAD: A = R. V C USTOMERS: N = R. V S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(2/V).dt = P[1].R.dt P[3].(3/V).dt = P[2].R.dt = (A 3 /3!). P[0] P[0] = 1 / (A j / j!) = e -A P[j] = ( A j / j! ). e -A S TATE P ROBABILITIES: P[1].(1/V).dt = P[0].R.dt P[2].(2/V).dt = P[1].R.dt P[3].(3/V).dt = P[2].R.dt = (A 3 /3!). P[0] P[0] = 1 / (A j / j!) = e -A P[j] = ( A j / j! ). e -A N O B LOCKING

41. A PPLICATION QUEUEING SYSTEMS Saad Haj Bakry A NSWERS: A = 10, E B [C,A] = 0.095 P [N<=5] = e -A ( A j / j! ) = P [N<=5] = 0.67 A NSWERS: A = 10, E B [C,A] = 0.095 P [N<=5] = e -A ( A j / j! ) = P [N<=5] = 0.67 P ROBLEM: INPUT: R = 1 [cus / (6) min] V = 4 [hour] FIND: Load : A P[N<=5] P ROBLEM: INPUT: R = 1 [cus / (6) min] V = 4 [hour] FIND: Load : A P[N<=5] M / M / infinity

42. M / M / 1 N ETWORK QUEUEING SYSTEMS Saad Haj Bakry N ETWORK D ESCRIPTION A PPLICATIONS P ERFORMANCE M EASURES

43. QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 Network C APACITY: C[i] : Channel C : Total C = C[i] (k) : Links C APACITY: C[i] : Channel C : Total C = C[i] (k) : Links L OAD: R[x,y]: Rate R [mps] : Total R = R(x,y) (n) : Nodes M [bits] : Average message length (R.M) [bps] : Load L OAD: R[x,y]: Rate R [mps] : Total R = R(x,y) (n) : Nodes M [bits] : Average message length (R.M) [bps] : Load F LOW: F[i] : Channels F [mps] : Total F = F[i] (k) : Links F LOW: F[i] : Channels F [mps] : Total F = F[i] (k) : Links F = z[x,y]. R[x,y] z[x,y]: Number of links in [x-y] path F = z[x,y]. R[x,y] z[x,y]: Number of links in [x-y] path N ETWORK D ESCRIPTION

44. A VERAGE P ER L INK : T = (1 / F) F[i].T[i] A VERAGE P ER L INK : T = (1 / F) F[i].T[i] QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 Network L INK [i]: T[i] = 1 / ( (C[i] / M) - F[i] ) P ERFORMANCE: D ELAY A VERAGE P ER P ATH : T [PATH] = z. T = (F/R).T C[i] >> F[i].M T[i] = 1 / (C[i] / M) = M / C[i] T [PATH] = (z / F). ( F[i]. (M/C[i] )

45. A PPLICATION: N ETWORK QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 Network B F A D E C 20 10 50 20 10 20 (1)(1) (2)(2) (3)(3) (6)(6)(4)(4) (5)(5) (7)(7)(8)(8) Full Duplex N Node ( L ) Link ID nn [kbps]: Capacity

46. A PPLICATION: L OAD QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 Network B F A D E C 47149 42389 542 4 4 23384 43331 53327 BFADEC FROM TO (message / sec)M = 800 bits

47. A PPLICATION: R OUTES QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 Network B F A D E C AEFAE ABFD AB BFBFEBFDBCBA FEFDFECFBFEA CEFCECDCBCBA DFDCEDCDFB DFBA EFECDECEFBEA BFADEC FROM TO ROUTES ABC

48. A PPLICATION: L INKS QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 Network 8 6 7 32 592520 8 EC 4 3 5 4815412.5106 CD 44712520 11 AE 20.82062.5 50 13EF 64 (*)222 12.5 108FD 40 67 252010BF 1 i 2 (F/C) [%] T[ms]C [mps] C [kbps] F [mps] LINK 56912520 14 AB 487725 20 12BC

49. A PPLICATION: A VERAGE QUEUEING SYSTEMS Saad Haj Bakry M / M / 1 Network A VERAGE P ATH: F = F[i] = 164 R = R[x,y] = 124 z = F / R = 1.32 A VERAGE P ATH: F = F[i] = 164 R = R[x,y] = 124 z = F / R = 1.32 A VERAGE M ESSAGE D ELAY: T = (z/F) (F[i].T[i]) T = 114 [ms] A VERAGE M ESSAGE D ELAY: T = (z/F) (F[i].T[i]) T = 114 [ms]

50. S UGGESTED W ORK QUEUEING SYSTEMS Saad Haj Bakry G ENERIC T OOLS: Evaluates the Performance of the Various Queuing Systems for General Use G ENERIC T OOLS: Evaluates the Performance of the Various Queuing Systems for General Use P RACTICAL E VALUATIONS: Related Queuing Case Studies P RACTICAL E VALUATIONS: Related Queuing Case Studies