1 Queuing Theory

2 Queuing theory is the study of waiting in lines or queues. Server Pool of potential customers Rear of queue Front of queue Line (or queue) of customers List of servers able to service the customers

3 Queuing Theory -- cont. What do we want to know about a queuing system? The average or expected wait time The percentage of customers who experience long wait times The probability that a customer must wait (The probability that all servers are busy) The average number of customers in the queue. The probability that servers are idle

4 Simulations Simulation is a method of evaluation without using methmatical models such as queuing theory. In general, a simulation is a computer- programmed model of something. The best test of an OS: the real marketplace. The next best test: create situation similar to the real world. Simulations play a key role in the development of complex system such as computer networks, databases, and OS

5 Queueing Theory Customer arrive at a queueing system randomly at time (arrival times) Poisson arrival process: the interarrival times are distributed exponentially: : constant average arrival rate = customer/unit time The # of arrivals / unit time is poisson distributed with mean = interarrival times = the time between successive arrivals t0t0 t1t1 t2t2 t3t3 t4t4 tntn

6 Example: arrival rate: one every two minutes The rate of one customer every two minutes = 0.5 customers per minute t P(t) The probability function does not tell us when customers will arrive. It does, however, provide information about the random arrival process.

7 y t (time) y = 1 P(  t) = Probability of an arbitrary interarrival time  being less than t for large for small t = 0

8 Server 1 Server C # of customers in service Service time # of customers in the queue Time spent in the queue Total time a customer spends in the queueing system Total # of customers in the queueing system Average arrival rate interarrival time N NsNs NqNq q S W 

9 The probability that exactly n customers will arrive in any time interval of length t is Let S k denotes the service time that the Kth arriving customer requires from the system. An arbitrary service time is referred to as S, and the distribution of service time is For random service with average service rate 

10 Queue Disciplines A queue discipline is the rule used for choosing the next customer from the queue to be serviced. Kendall notation A/B/c A is the interarrival time distribution B is the service time distribution c is the number of servers A and B may be –GI for general independent interarrival time –G for general service time –M for exponential interarrival or service time distribution –D for deterministic interarrival or service time distribution M/M/1 M/M/c

11 Traffic Intensity A measure of that system's ability to service its customer effectively It is useful for determining the minimum number of identical servers a system will need in order to service its customer without its queue becoming indefinitly large or having to turn customer away. Ex: E(s) = 17 sec E(  ) = 5 sec u = 17/5 = 3.4 need at least 4 servers arrival rate service rate

12 Case Studies A shared Laser printer An average of 64 requests occurring at random times during eight-hour day. Each request require an average of about 5 minutes to print. Receiving compaints from employees that they must wait nearly half an hour for their printout. 8 requests/hour  = 2/15 requests/minute 12 requests/hour  = 1/ 5 request/minute

13 Server Utilization Server utilization  is defined as the traffic intensity per server is the probability that a particular server is busy this is approximately the fraction of time that each server is in use For single-server system, u =  

14 Probability of All Server Idle When c= 1 (1 - 2/3 = 1/3)

15 Probability of All Server Busy Erlang’s C Formula: When c = 1 (2/3)

16 Expected Number of Customers The expected number of customers in the queue: When c = 1   

17 Expected Wait Time Little’s Formula Expected wait time When c = 1 (2/15)* 10 = 4/3 [(2/3)/0.2]/(1-2/3) = 10 minutes

18 90th Percentile Wait Time 90% of the customers wait less than When c = 1 ln(10*2/3)/(1/5 - 2/15) = 28.4 minutes

19 Case Study 2 Master Scheduler Computer running simulation programs requires a lot of CPU time and scheduled on a FIFO basis. Generally submit about 100 programs per day The programs require an average of about one hour of CPU time. 100 requests/day  = 100/24 = 4.2 requests/hour 1 requests/hour  = 1 request/hour

20 Example 1 The election of the President of Student Association （學生會） is just finished in Tamkang University. There are 27,000 students and the average rate of voting is 15%. The official voting period was 5 days, from 8:00 to 17:00. The association prepared only one counter for everyone to vote and each voting requires half minute to complete. Use the equation provided to analyze the condition of student getting in-line and wait for the voting. You must explain the meaning of each equation. (ln(2.5)=0.9, ln(7.5)=2, ln(15)=2.7)

21 = 27000*.15/5*9 = 90 students/hour  = 120 students/hour  = /  = 3 / 4 (Server utilization) P 0 = 1 -  = 1/4 (probability of all server idle) C = 3/4 (probability of all server busy) L q = (3/4)2 / 1 – 3/4 = 9/4 (Expected number of customers) W q = [(3/4)/120] / 1 – 3/4 = 1/40 hour (Expected wait time)  q = ln(10* 3/4) / 120 – 90 = 1/15 hour (90th percentile wait time)

22 Example 2 There are 48 programmers per day, in average, come in to use the mainframe computer in the computing center. Each programmer uses the computer for about 20 minutes. Use the equation provided to analyze the usage of the computer. You must explain the meaning of each equation.

23 = 48/24 = 2 programmer/hour  = 3 programmer/hour  = /  c = 2 / 3 (Server utilization) P 0 = 1 -  = 1 / 3 (probability of all server idle) C = 2 / 3 (probability of all server busy) L q = (2/3) 2 / 1 – 2/3 = 4 / 3 (Expected number of customers) W q = [(2/3)/3] / 1 – 2/3 = 2 / 3 hour (Expected wait time)  q = ln(10* 2/3) / 3 – 2 = 1.9 hour (90th percentile wait time)

24 Example 3 There are 36 programmers per day, in average, come in to use the mainframe computer in the computing center. Each programmer uses the computer for about 15 minutes. Use the equation provided to analyze the usage of the computer. You must explain the meaning of each equation.