1. Queuing Theory

2.  Queuing Theory deals with systems of the following type:  Typically we are interested in how much queuing occurs or in the delays at the servers. Queuing Theory Input Process Server Process(es) Output

3. Queuing Theory Notation A standard notation is used in queuing theory to denote the type of system we are dealing with. Typical examples are: – M/M/1Poisson Input/Poisson Server/1 Server – M/G/1Poisson Input/General Server/1 Server – D/G/nDeterministic Input/General Server/n Servers – E/G/  Erlangian Input/General Server/Inf. Servers The first letter indicates the input process, the second letter is the server process and the number is the number of servers. (M = Memoryless = Poisson)

4. 4 Definition of a queueing system Customer arrivals Departure of impatient customers Departure of served customers A queueing system can be described as follows: "customers arrive for a given service, wait if the service cannot start immediately and leave after being served" The term "customer" can be men, products, machines,...

5. 5 History of queueing theory The theory of queueing systems was developed to provide models for forecasting behaviors of systems subject to random demand. The first problems addressed concerned congestion of telephone traffic (Erlang, "the theory of probabilities and telephone conversations ", 1909) Erlang observed that a telephone system can be modeled by Poisson customer arrivals and exponentially distributed service times Molina, Pollaczek, Kolmogorov, Khintchine, Palm, Crommelin followed the track

6. 6 Interests of queueing systems Queuing theory found numerous applications in: – Trafic control (communication networks, air traffic, …) – Planing (manufacturing systems, computer programmes, …) – Facility dimensioning (factories,...)

7. 7 Classification of queueing systems

8. 8 Characteristics of simple queueing systems Queueing systems can be characterized with several criteria: Customer arrival processes Service time Service discipline Service capacity Number of service stages

9. 9 Notation of Kendall The following is a standard notation system of queueing systems T/X/C/K/P/Z with – T: probability distribution of inter-arrival times – X: probability distribution of service times – C: Number of servers – K: Queue capacity – P: Size of the population – Z: service discipline

10. 10 Customer arrival process T/X/C/K/P/Z T can take the following values: – M : markovian (i.e. exponential) – G : general distribution – D : deterministic – Ek : Erlang distribution – … If the arrivals are grouped in lots, we use the notation T[X] where X is the random variable indicating the number of customers at each arrival epoch – P{X=k} = P{k customers arrive at the same time} Some arriving customers can leave if the queue is too long

11. 11 Service times T/X/C/K/P/Z X can take the following values: – M : markovian (i.e. exponential) – G : general distribution – D : deterministic – Ek : Erlang distribution – … Erlang distribution E k with parameter  k exponential servers with parameter 

12. 12 Number of servers T/X/C/K/P/Z In simple Queuing Systems, servers are identical

13. 13 Queue capacity T/X/C/K/P/Z Loss of customers if the queue is full Capacity K

14. 14 Size of the population T/X/C/K/P/Z The size of the population can be either finite or infinite For a finite population, the customer arrival rate is a function of the number of customers in the system: (n).

15. 15 Service discipline T/X/C/K/P/Z Z can take the following values: FCFS or FIFO : First Come First Served LCFS or LIFO : Last Come First Served RANDOM : service in random order HL (Hold On Line) : when an important customer arrives, it takes the head of the queue PR ( Preemption) : when an important customer arrives, it is served immediately and the customer under service returns to the queue PS (Processor Sharing) : All customers are served simultaneously with service rate inversely proportional to the number of customers GD (General Discipline)

16. Chapter 30 Introduction to Queueing Theory 16  = interarrival time = mean arrival rate = 1/E[  ] s = service time per job  = mean service rate per server = 1/E[s] (total service rate for m servers is m  ) Time Previous arrival Arrival Begin Service End Service  ws n q = number of jobs waiting to receive service. n s = number of jobs receiving service n = number of jobs in system n = n q + n s r = response time = w + s w = waiting time r Note that all of these variables are random variables except for and .

17. Chapter 30 Introduction to Queueing Theory 17 Rules for All Queues Stability Condition – If the number of jobs becomes infinite, the system is unstable. For stability, the mean arrival rate less than the mean service rate. < m  – Does not apply to finite buffer system or the finite population systems  They are always stable. Finite population: queue length is always finite. Finite buffer system: arrivals are lost when the number of jobs in the system exceed the system capacity.

18. Chapter 30 Introduction to Queueing Theory 18 Rules for All Queues Number in System vs. Number in Queue – n = n q + n s – E[n] = E[n q ]+E[n s ] – Also, if the service rate of each server is independent of the number in queue  Cov(n q,n s ) = 0  Var[n] = Var[n q ]+Var[n s ]

19. Chapter 30 Introduction to Queueing Theory 19 Rules for All Queues Number vs. Time (Little’s law) – If jobs are not lost due to buffer overflow, the mean jobs is related to its mean response time as follows: mean number of jobs in system = arrival rate x mean response time – Similarly mean jobs in queue = arrival rate x mean waiting time – For finite buffers can use effective arrival rate, that is, the rate of jobs actually admitted to the system.

20. Chapter 30 Introduction to Queueing Theory 20 Rules for All Queues Time in System vs. Time in Queue – Time spent in system, response time, is the sum of waiting time and service time r = w + s – In particular: E[r] = E[w] + E[s] – If the service rate is independent of jobs in queue  Cov(w,s) = 0  Var[r] = Var[w] + Var[s]

21. Chapter 30 Introduction to Queueing Theory 21 Proof of Little’s Law Mean jobs in system = arrival rate x mean response time Arrival timeDeparture time job104 job236 job357

22. Chapter 30 Introduction to Queueing Theory 22 Proof of Little’s Law J: total time spent inside the system by all three jobs (the hatched area) 30.4 (a): Mean time spent in the system = J/N 30.4 (b): Mean number in the system = J/T = J/N * N/T = Mean time spent in the system * Mean arrival rate

23. Chapter 30 Introduction to Queueing Theory 23 Applying Little’s Law Example: – A disk server satisfies an I/O request in average of 100 msec. I/O rate is about 100 requests/sec. What is the mean number of requests at the disk server? – Mean number at server = arrival rate x response time = (100 requests/sec) x (0.1 sec) = 10 requests

24. Chapter 30 Introduction to Queueing Theory 24 Types of Stochastic Processes A stochastic process is a family of random variables indexed by the parameter t, such as time. e.g.) Number of jobs at CPU of computer system at time t is a random variable n(t) Time and state can be discrete or continuous

25. Chapter 30 Introduction to Queueing Theory 25 Types of Stochastic Processes Markov Process – If future states depend only on the present and are independent of the past (memoryless) then called markov process – A discrete state Markov Process is a Markov chain – M/M/m queues can be modeled using Markov process time spent by a job – Markov process the number of jobs - Markov chain

26. Chapter 30 Introduction to Queueing Theory 26 Types of Stochastic Processes Birth-Death Process – The Markov chain in which transitions restricted to neighboring states. – Can represent states by integers, s.t. process in state n can only go to state n+1 or n-1 – Arrival (birth) causes state to change by +1 and departure after service (death) causes state to change by –1 – Only if arrive individually, not in bulk

27. Chapter 30 Introduction to Queueing Theory 27 Types of Stochastic Processes Poisson Processes – If inter arrival times are IID and exponentially distributed, then number of arrivals over interval [t,t+x] has a Poisson distribution  Poisson Process – Popular in queuing theory because arrivals are then memory less

28. Chapter 30 Introduction to Queueing Theory 28 Properties of Poisson Process (a)Merging k Poisson streams with mean rate i gives another Poisson stream with mean rate =  i (b) If Poisson stream split into k substreams with probability p i, each substream is Poisson with mean rate p i

29. Chapter 30 Introduction to Queueing Theory 29 Properties of Poisson Process (c) If arrivals to single server with exponential service times are Poisson with mean, departures are also Poisson with mean, if ( <  ) (d) Same relationship holds for m servers as long as total arrival rate less than total service rate

30. Chapter 30 Introduction to Queueing Theory 30 Types of Stochastic Processes Poisson Processes Birth-death Processes Markov Processes