2 Queueing Systems Queue Queueing system a line of waiting customers who require service from one or more service providers.Queueing systemwaiting room + customers + service provider?ArrivalsCustomersQueueServer(s)Departures
3 Queuing ModelsWidely used to estimate desired performance measures of the systemProvide rough estimate of a performance measureTypical measuresServer utilizationLength of waiting linesDelays of customersApplicationsDetermine the minimum number of servers needed at a service centerDetection of performance bottleneck or congestionEvaluate alternative system designs
4 Introduction Examples are: Example questions: – waiting to pay in the supermarket– waiting at the telephone for information– planes circle before they can landExample questions:– what is the average waiting time of a customer?– how many customers are waiting on average?– how long is the average service time?– what is the chance that one of the servers has nothing to do?
5 2. Service time distribution 1. Arrival ProcessQueue5. Customer Population4. Server Capacity=# of seats for customer waiting for service +# of servers3. Number of ServersQueueing Notation
6 Parameters of a queuing system The behavior of a queuing system is dependent on:Arrival process (l and distribution of interarrival times)Service process (m and distribution of service times)Number of serversCapacity of the systemSize of the population for this system
7 Kendall notation A/S/m/B/K/SD A: arrival process S: service time distributionm: number of serversB: number of buffers (system capacity)K: population sizeSD: service discipline
8 Number of Servers Number of servers available Single Server Queue Multiple Server Queue
9 Service Disciplines First-come-first-served(FCFS) Last-come-first-served(LCFS)Shortest processing time first(SPT)Shortest remaining processing time first(SRPT)Shortest expected processing time first(SEPT)Shortest expected remaining processing time first(SERPT)Biggest-in-first-served(BIFS)Loudest-voice-first-served(LVFS)
10 Common Distributions M : Exponential Ek : Erlang with parameter k Hk : Hyperexponential with parameter k(mixture of k exponentials)D : Deterministic(constant)G : General(all)f(t)tmsf(t)1/mtmGeneralExponential
11 ExampleM/M/3/20/1500/FCFSTime between successive arrival is exponentially distributed.Service times are exponentially distributed.Three servers.20 buffers = 3 service + 17 waitingAfter 20, all arriving jobs are lost.Total of 1500 jobs that can be serviced.Service disciple is fist-come-first served.
12 Defaults Infinite buffer capacity Infinite population size FCFS service disciple=> The first three of the six parameter are sufficientG/G/1=G/G/1/∞/∞/FCFSAssume only individual arrivals (no bulk arrivals)
13 Arrival process Arrival times: Interarrival times: tj form a sequence of Independent and Identically Distributed (IID) random variablesThe most common arrival process: Poisson arrivalsInter-arrival times are exponential + IID Poisson arrivals
14 Service time distribution Amount of time each customer spends at the serverAgain, usually assume IIDMost commonly used distribution is the exponential distribution.Distribution: M, E, H, or G
15 Key VariablesServiceArrivalRate (Average Numberin Queue (Nq )Average Waitin Queue (w )Rate (Departuret = Inter-arrival time = time between two successive arrivalsl = Mean arrival rate = 1/E[t] May be a function of the state of the system, e.g., number of jobs already in the systems = Service time per jobm = Mean service rate per server = 1/E[s]Total service rate for m servers is mm
16 Key Variables (cont’d) nq = Number of jobs waitingns = Number of jobs receiving servicen = Number of jobs in the system. This is also called queue lengthNote: Queue length includes jobs currently receiving service as well as those waiting in the queuer = Response time or the time in the system = time waiting + time receiving servicew = Waiting time = Time between arrival and beginning of service
17 Key Variables 1 2 m Previous Arrival Arrival Begin Service End Service twsrnnqnslTimeNote that all of these variables are random variablesexcept for and .
18 Rules for All Queues Stability Condition If the number of jobs becomes infinite, the system is unstable. For stability, the mean arrival rate less than the mean service rate. < mDoes not apply to finite buffer system or the finite population systems They are always stable.Finite population: queue length is always finite.Finite buffer system: arrivals are lost when the number of jobs in the system exceed the system capacity.
19 Rules for All QueuesNumber in System versus Number in Queue: n = nq+ ns Notice that n, nq, and ns are random variables.E[n]=E[nq]+E[ns]If the service rate is independent of the number in thequeue,Cov(nq,ns) = 0
20 Rules for All QueuesNumber versus Time: If jobs are not lost due to insufficient buffers, Mean number of jobs in the system = Arrival rate Mean response timeSimilarly, Mean number of jobs in the queue = Arrival rate Mean waiting timeThis is known as Little's law.
21 Rules for All Queues E[r] = E[w] + E[s] Time in System versus Time in Queue r = w + s r, w, and s are random variablesE[r] = E[w] + E[s]If the service rate is independent of the number of jobs in the queue, Cov(w,s)=0
22 Little's LawMean number in the system = Arrival rate Mean response timeThis relationship applies to all systems or parts of systems in which the number of jobs entering the system is equal to those completing serviceBased on a black-box view of the system:In systems in which some jobs are lost due to finite buffers, the law can be applied to the part of the system consisting of the waiting and serving positionsBlack BoxArrivalsDepartures
23 Application of Little's Law ArrivalsDeparturesApplying to just the waiting facility of a service centerMean number in the queue = Arrival rate Mean waiting timeSimilarly, for those currently receiving the service, we have:Mean number in service = Arrival rate Mean service time
24 ExampleA monitor on a disk server showed that the average time to satisfy an I/O request was 100 milliseconds. The I/O rate was about 100 requests per second. What was the mean number of requests at the disk server? Using Little's law:Mean number in the disk server= Arrival rate Response time= 100 (requests/second) (0.1 seconds)= 10 requests
25 Stochastic ProcessesA stochastic process is a sequence X1,X2, of random variables indexed by the parameter t, such as time.For example, number of jobs at CPU of computer system at time t is a random variable X(t)Time and state can be discrete or continuous.A queuing system is characterized by three elements:A stochastic input processA stochastic service mechanism or processA queuing discipline
26 Types of Stochastic Processes PoissonProcessesBirth-deathMarkov
27 Types of Stochastic Processes Markov ProcessIf future states depend only on the present and are independent of the past (memoryless) then called markov processA discrete state Markov Process is a Markov chainM/M/m queues can be modeled using Markov processThe time spent by a job in such a queue is a Markov process and the number of jobs in the queue is a Markov chain
28 Types of Stochastic Processes Birth-Death ProcessThe discrete space Markov processes in which the transitions are restricted to neighboring statesProcess in state n can change only to state n+1 or n-1ExampleThe number of jobs in a queue with a single server and individual arrivals(not bulk arrivals)12j-1jj+1…l0l1l2lj-1ljlj+1m1m2m3mjj+1 j+2
30 M/M/1 QueueM/M/1 queue is the most commonly used type of queueUsed to model single processor systems or to model individual devices in a computer systemAssumes that the interarrival times and the service times are exponentially distributed and there is only one serverNo buffer or population size limitations and the service discipline is FCFSNeed to know only the mean arrival rate l and the mean service rate mState = number of jobs in the system12J-1JJ+1…lm
31 Results for M/M/1 Queue Birth-death processes with Probability of n jobs in the system:
32 M/M/1 Queue Parameters: Traffic intensity: Stability Condition: Traffic intensity r must be less than oneProbability of zero jobs in the system: p0 = 1- rProbability of n jobs in the system:Mean number of jobs in the system:
33 M/M/1 Queue (cont’d) Variance of number of jobs in the system: Probability of k jobs in the queue =