Queuing Theory Simulation & Modeling

The Queuing Model Use Queuing models to Queue Server Queuing System Use Queuing models to Describe the behavior of queuing systems Evaluate system performance

Applications of Queuing Theory Telecommunications Computer Networks Predicting computer performance Health services (eg. control of hospital bed assignments) Airport traffic, airline ticket sales Layout of manufacturing systems.

Example application of queuing theory In many stores and banks, we can find: multiple line/multiple checkout system → a queuing system where customers wait for the next available cashier We can prove using queuing theory that : throughput improves/increases when queues are used instead of separate lines

The Queuing Model A Queue System is characterized by Server Queuing System A Queue System is characterized by Queue (Buffer): with a finite or infinite size The state of the system is described by the Queue Size Server: with a given processing speed Events: Arrival (birth) or Departure (death) with given rates

Example application of queuing theory

Open Queueing Networks Open queueing network: external arrivals and departures Number of jobs in the system varies with time Throughput = arrival rate Goal: To characterize the distribution of number of jobs in the system

Closed Queueing Networks Closed queueing network: No external arrivals or departures Total number of jobs in the system is constant “OUT” is connected back to “IN” Throughput = flow of jobs in the OUT-to-IN link Number of jobs is given, determine the throughput

Queuing theory for studying networks View network as collections of queues FIFO data-structures Queuing theory provides probabilistic analysis of these queues Examples: Average length Average waiting time Probability queue is at a certain length Probability a packet will be lost

Basic Components of a Queue 2. Service time distribution 1. Arrival process 6. Service discipline 5. Customer Population 4. Waiting positions 3. Number of servers Example: students at a typical computer terminal room with a number of terminals. If all terminals are busy, the arriving students wait in a queue.

Characteristics of queuing systems Arrival Process The distribution that determines how the tasks arrives in the system. Service Process The distribution that determines the task processing time Number of Servers Total number of servers available to process the tasks Storage capacity: are buffer finite or infinite?

Specification of Queueing Systems Operating policies Customer class differentiation are all customers treated the same or do some have priority over others? Scheduling/Queueing policies which customer is served next Admission policies which/when customers are admitted

Service Disciplines First-Come-First-Served (FCFS) Last-Come-First-Served (LCFS) Last-Come-First-Served with Preempt and Resume (LCFS-PR) Round-Robin (RR) with a fixed quantum. Small Quantum  Processor Sharing (PS) Infinite Server: (IS) = fixed delay Shortest Processing Time first (SPT) Shortest Remaining Processing Time first (SRPT) Shortest Expected Processing Time first (SEPT) Shortest Expected Remaining Processing Time first (SERPT). Biggest-In-First-Served (BIFS) Loudest-Voice-First-Served (LVFS)

20 Buffers = 3 service + 17 waiting Example Three servers 20 Buffers = 3 service + 17 waiting After 20, all arriving jobs are lost Total of 1500 jobs that can be serviced Service discipline is first-come-first-served

The Queuing Times Queue Server Queuing System Queuing Time Service Time Response Time (or Delay)

The Little’s Law The long-term average number of customers in a stable system N, is equal to the long-term average arrival rate, λ, multiplied by the long-term average time a customer spends in the system, T.

Little's Law Mean number in the system = Arrival rate  Mean response time This relationship applies to all systems or parts of systems in which the number of jobs entering the system is equal to those completing service Based on a black-box view of the system: In systems in which some jobs are lost due to finite buffers, the law can be applied to the part of the system consisting of the waiting and serving positions Black Box Arrivals Departures

Little’s Law a (t): the process that counts the number of arrivals up to time t. d (t): the process that counts the number of departures up to time t. N(t)= a(t)- d(t) a(t) d(t) # Arrivals Area J(t) N(t) Time t Average arrival rate (up to t) λt= a(t)/t Average time each customer spends in the system (Response Time) Tt= J(t)/a(t) Average number in the system Nt= J(t)/t

Rules for All Queues (cont’d) Number versus Time: If jobs are not lost due to insufficient buffers, Mean number of jobs in the system = Arrival rate  Mean response time Similarly, Mean number of jobs in the queue = Arrival rate  Mean waiting time This is known as Little's law. Time in System versus Time in Queue r = w + s r, w, and s are random variables E[r] = E[w] + E[s]

Example The average queue length in the computer system was observed to be: 8.88, 3.19, and 1.40 jobs at the CPU, disk A, and disk B, respectively. What were the response times of these devices? the device throughputs were determined to be: The new information given in this example is: Using Little's law, the device response times are:

Operational Quantities Black Box Quantities that can be directly measured during a finite observation period T = Observation interval Ai = number of arrivals Ci = number of completions Bi = busy time Bi

Utilization Law This is one of the operational laws Operational laws are similar to the elementary laws of motion For example, Notice that distance d, acceleration a, and time t are operational quantities. No need to consider them as expected values of random variables or to assume a distribution

Example Consider a network gateway at which the packets arrive at a rate of 125 packets per second and the gateway takes an average of two milliseconds to forward them Throughput Xi = Exit rate = Arrival rate = 125 packets/second Service time Si = 0.002 second Utilization Ui= Xi Si = 125  0.002 = 0.25 = 25% This result is valid for any arrival or service process. Even if inter-arrival times and service times to are not IID random variables with exponential distribution

Forced Flow Law Relates the system throughput to individual device throughputs In an open model, System throughput = # of jobs leaving the system per unit time In a closed model, System throughput = # of jobs traversing OUT to IN link per unit time If observation period T is such that Ai = Ci  Device satisfies the assumption of job flow balance Each job makes Vi requests for i-th device in the system If the job flow is balanced and C0 is # of jobs traversing the outside link => Ci – # of jobs visiting the i-th device: Ci = C0 Vi or Vi =Ci/C0 Vi is called visit ratio

Forced Flow Law (cont’d) System throughput:

Forced Flow Law (cont’d) Throughput of ith device: In other words: This is the forced flow law

Example In a timesharing system, accounting log data produced the following profile for user programs Each program requires five seconds of CPU time, makes 80 I/O requests to the disk A and 100 I/O requests to disk B Average think-time of the users was 18 seconds From the device specifications, it was determined that disk A takes 50 milliseconds to satisfy an I/O request and the disk B takes 30 milliseconds per request With 17 active terminals, disk A throughput was observed to be 15.70 I/O requests per second We want to find the system throughput and device utilizations

Example (cont’d) Since the jobs must visit the CPU before going to the disks or terminals, the CPU visit ratio is:

Example (cont’d) Using the forced flow law, the throughputs are: Using the utilization law, the device utilizations are:

Example A monitor on a disk server showed that the average time to satisfy an I/O request was 100 milliseconds. The I/O rate was about 100 requests per second. What was the mean number of requests at the disk server? Using Little's law: Mean number in the disk server = Arrival rate  Response time = 100 (requests/second) (0.1 seconds) = 10 requests