1. Page 1 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Things I Thought I Knew About Queueing Theory, but was Wrong About: Part 1, Multiserver Queues Alan Scheller-Wolf Tepper School of Business Carnegie Mellon University Joint work with Karl Sigman, Rein Vesilo, and Michele Dufalla

2. Page 2 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 A Great Day As a graduate student, one of the greatest things that can happen is the following: Student: “I can prove that XXX is true!” Advisor: “That can’t be right, everyone knows YYY is true.” Student: “I know, but I can prove it.” Advisor: “There must be a bug, show me.” Time passes... Advisor: “Huh, you are right, XXX is true!”

3. Page 3 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 My Goals Take you through “my great day” as a student Help you understand why everyone believed YYY Help you understand, mathematically, why XXX is true Help you see why, intuitively, XXX “has to be true” Discuss what general insights this gives

4. Page 4 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Mean Delay in GI/GI/1 Queues i.i.d. arrivals ~ T rate  i.i.d. service ~ S  rate  Jobs are served in a FIFO (FCFS) fashion Delay: Time customers spend in queue Question: What are conditions for mean (stationary) delay to be finite? Kiefer and Wolfowitz (K&W, 1956)    E[D] finite iff E[S 2 ] finite Single Server Queue S should be “heavy tailed” -- think Pareto distribution

5. Page 5 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Why is E[S 2 ] Necessary (for Single-Server Queues)? The inspection paradox  A randomly arriving job sees an equilibrium service time S e (or the excess of a service time)  Length biasing causes the tail of S e [P(S e >x)] to behave as the integrated tail of S – you are more likely to “see” longer service times  E[S e ] is finite iff E[S 2 ] is finite The upshot: If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S 2 ]. The arriving job can’t enter service until this job is done.

6. Page 6 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Why is E[S 2 ] Sufficient (for Single-Server Queues)? (I) The Lindley Equation (Lindley, 1952)  D n+1 = (D n +S n -T n ) +, where S n is the service time of customer n and T n is the interarrival time between customers n and n+1  Why?  Let’s say customer n arrives at time   Then customer n leaves service at time  +D n +S n  Customer n+1 arrives at time  T n  Customer n+1 enters service at time min{  +D n +S n,  T n }  Customer n+1’s delay is min{  +D n +S n,  T n } – (  T n ) This is a reflected random walk

7. Page 7 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Why is E[S 2 ] Sufficient (for Single-Server Queues)? (II) Reflected Random Walks: For Z n+1 = (Z n +X n ) +, E[Z 2 ] is finite if:  E[(X + ) 2 ] is finite  E[X n ] < -  for all n and on every sample path For our GI/GI/1 queue:  E[(X + ) 2 ] = E[S 2 ] is finite  E[X n ] = E[S] - E[T] < -  ; for all n and on every sample path, since 

8. Page 8 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Mean Delay in GI/GI/k Queues i.i.d. arrivals ~ T rate  i.i.d. service ~ S  rate  Question: What are conditions for mean stationary delay to be finite? K&W, 1956   k   E[D] finite if E[S 2 ] finite Multi-Server Queue

9. Page 9 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Why is E[S 2 ] Sufficient? Split the GI/GI/k into k GI/GI/1 queues  This performs no better than the original system Route arriving jobs to queues in cyclic (or round robin) fashion Each GI/GI/1 queue has finite mean delay Therefore the original GI/GI/k queue has finite mean delay as well

10. Page 10 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 What about Necessity? “Everyone knew” that  k  E[S 2 ] finite were necessary as well. This had even appeared in the literature.

11. Page 11 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Pictorially: Requirement for finite mean delay 1 2 Servers 1234 ∞

12. Page 12 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Back to the GI/GI/1 Why is finite E[S 2 ] Necessary? The inspection paradox  If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S 2 ]. The arriving job can’t enter service until this job is done. Does this logic hold for the GI/GI/k? NO! Can we capture this effect mathematically?

13. Page 13 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Lindley Equation for GI/GI/1 Our logic for GI/GI/1  Let’s say customer n arrives at time   Then customer n leaves service at time  +D n +S n  Customer n+1 arrives at time  T n  Customer n+1 enters service at time min{  +D n +S n,  T n }  Customer n+1’s delay is min{  +D n +S n,  T n } – (  T n ) D n+1 = (D n +S n -T n ) +

14. Page 14 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Lindley Equation for GI/GI/k For GI/GI/k  Let’s say customer n arrives at time   Then customer n leaves service at time  +D n +S n  Customer n+1 arrives at time  T n  Customer n+1 enters service at time min{  +D n +S n,  T n } NO!

15. Page 15 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 When Does Customer n+1 Enter Service? (I) Customer n arrives at time  Customer n leaves service at time  +D n +S n n n+1 j

16. Page 16 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 When Does Customer n+1 Enter Service? (II) When does customer j leave service?  Let’s say that when customer n enters service, she looks over at the other server, and observes the amount of time customer j has left  Let’s call this amount of time U n  Then customer j leaves service at time  +D n +U n n n+1 j If there were multiple other servers, U n would be the minimum remaining time

17. Page 17 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 When Does Customer n+1 Enter Service? (III) n n+1 j Customer n leaves service at time  +D n +S n Customer j leaves service at time  +D n +U n So the next server comes free at time  +D n +min{S n,U n } =  +D n +P n

18. Page 18 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Lindley Equation for GI/GI/k For GI/GI/k: D n+1 = (D n +P n -T n ) + How big of difference could one letter make? We have replaced S n with P n

19. Page 19 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Back to Reflected Random Walks For our GI/GI/k queue, for finite mean delay it is sufficient that:  E[(X + ) 2 ] = E[P 2 ] is finite  E[X n ] = E[P n ] - E[T n ] < -  ; for all n and on every sample path How does this help? We don’t know the first thing about P!

20. Page 20 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 When is E[P 2 ] Finite? E[P 2 ] = E[min{S,U} 2 ] = E[min{S,U} 3/2 min{S,U} 1/2 ] < E[S 3/2 U 1/2 ] = E[S 3/2 ] E[U 1/2 ] And E[U 1/2 ] < E[W 1/2 ] And E[W 1/2 ] is finite iff E[S 3/2 ] (K&W, 1956) So for a GI/GI/2 queue, E[P 2 ] is finite if E[S 3/2 ] is finite! S is independent of U W is the total work in system

21. Page 21 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 What about Negative Drift? E[X n ] = E[P n ] - E[T n ] = E[min{S n,U n }] - E[T n ] < E[S n ] - E[T n ] = E[S] - E[T] This seems like a really weak bound, but is in fact the best we can do, since this must hold on all sample paths and for all n; U n could be arbitrarily large on a sample path I have negative drift for all n on every sample path if E[S] < E[T] (  < ½) but not otherwise

22. Page 22 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Finite Mean Delay in GI/GI/2 (I) So it appears that: Finite E[S 3/2 ] is sufficient for finite mean delay in GI/GI/2 queues if E[S] < E[T] Finite E[S 3/2 ] is not sufficient for finite mean delay in GI/GI/2 queues if E[T] < E[S] < 2E[T] Does this make sense? YES!

23. Page 23 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Finite Mean Delay in GI/GI/2 (II) The inspection paradox  If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S 2 ]  The mean time this job has been in service also is on the order of E[S 2 ]  The key question is what has been going on while this long job has been in service

24. Page 24 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Finite Mean Delay in GI/GI/2 (III) If E[T] < E[S] < 2E[T]:  While a single extremely large job is in service the remaining server cannot keep up with the workload, and the system “crashes” If E[S] < E[T]:  A single extremely large job is not typically enough to crash the system – the remaining server can keep up with the workload  It takes two extremely large jobs to crash the system

25. Page 25 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Redefining  For a k-server queue  is traditionally defined as  k  This is WRONG! If instead   E[S] < E[T] means  and we have a “spare server” to absorb the workload if a long job arrives  E[T] < E[S] < 2E[T] means 1<  and we have no spare capacity  denotes the number of servers needed to handle the mean load on the system

26. Page 26 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 What about GI/GI/3? If 2<  <3 (no spare servers): E[D] is finite if E[S 2 ] is finite If 1<  <2 (one spare server): E[D] is finite if E[S 3/2 ] is finite If 0<  <1 (two spare servers): E[D] is finite if E[S 4/3 ] is finite

27. Page 27 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 What about GI/GI/k? If k-1<  <k (no spare servers): E[D] is finite if E[S 2 ] is finite If k-2<  <k-1 (one spare server): E[D] is finite if E[S 3/2 ] is finite If j<  <j+1 (k-j-1 spare servers): E[D] is finite if E[S (k+1-j)/(k-j) ] is finite

28. Page 28 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Pictorially: Fixed Number of Spare Servers Requirement for finite mean delay 1 2 Servers 1234 ∞ No spare servers One spare server Two spare servers 3/2 4/3 Diminishing benefits For moment conditions, 20-server queue with load 18.5 is the same as a 2-server queue with load 0.5 Pictorially:

29. Page 29 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Pictorially: As Spare Servers Grow Requirement for finite mean delay 1 2 Servers 1234 ∞ 3/2 4/3 As the number of spare servers approaches infinity, the moment conditions approach those of an infinite-server queue, as they should

30. Page 30 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Other Results (I) Higher Moments:  If j<  <j+1 (k-j-1 spare servers): E[D r ] is finite if E[S(k+r-j)/(k-j) ] is finite (r has replaced 1 in the previous formula)  If you have just two spare servers, E[D 2 ] is finite if E[S 5/3 ] is finite – you get finite variance of delay without finite variance of service time

31. Page 31 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Other Results (II) Necessary Conditions:  The sufficient conditions above are also provably necessary for all but a small set of service time distributions

32. Page 32 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Other Results (III) Integral Load:  Drift has been crucial in our analysis – if  E[S 3/2 ] is sufficient for finite mean delay; if  we require E[S 2 ]  What if   We have been able to show that E[S 3/2 ] is not sufficient, but have not been able to get close to showing E[S 2 ] is necessary  We are still working

33. Page 33 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Insights Delay moments in multi-server queues with heavy- tailed service times depend on the moments of S and the number of spare servers   is most usefully defined as for multi- server queues Adding spare serves helps with delay moments, but in diminishing fashion This works because under heavy-tailed service times queues build up due to large jobs blocking servers

34. Page 34 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Open Problems 1.Dependencies:  Everything we showed was for GI/GI/k; what if interarrival and/or service times are not i.i.d?  Our “Lindley” equation for the GI/GI/k: D n+1 = (D n +P n -T n ) + also holds for a special type of GI/G/1 queue. (It actually defines a GI/G/1 queue.) So there must be a relationship/duality here.

35. Page 35 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Open Problems (II) 2.Bounds:  As of now, the best analytical bounds on multi- server delay under heavy-tailed service time are in general very poor  These cannot, in general, be easily simulated  See work by Daley, Scheller-Wolf, Vesilo

36. Page 36 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Open Problems (III) Tail Asymptotics  Moment conditions for delay are really just a proxy for what we really want to know: How the tail of the delay distribution behaves  See work by Boxma, Foss, Korshunov, Whitt, Zwart

37. Page 37 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 References Scheller-Wolf, A. (2003). Necessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues: Why s Slow Servers are Better than One Fast Server for Heavy-Tailed Systems. Operations Research 51, No. 5, 748-758. Scheller-Wolf, A. and R. Vesilo (2006). Structural Interpretation and Derivation of Necessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues. Queueing Systems, 54, No. 3 221-232.

38. Page 38 Alan Scheller-Wolf Lunteren, The Netherlands January 15, 2013 Dank U! Any Questions?