1. 10/12/1999(c) Ian Davis1 Predicting performance Topics: –Operational analysis of network models –Markov models

2. 10/12/1999(c) Ian Davis2 Operational analysis Material drawn from: ACM Computing Surveys –Special issue on queuing network models –Volume 10, Number 3, September 1978 –P.J. Denning & J. P. Buzen –The operational analysis of network models

3. 10/12/1999(c) Ian Davis3 Basic ABC of quantities A:Total number of [A]rrivals B:Total time system [B]usy C:Total number of [C]ompletions T:Total time spent monitoring above

4. 10/12/1999(c) Ian Davis4 Basic derived quantities Arrival rate λ –A/T(tasks/second) Departure rate X –C/T(tasks/second) Server utilization U –B/T(fraction) Mean service time S per task –B/C(seconds/task)

5. 10/12/1999(c) Ian Davis5 Utilization law U = XS B/T = (C/T) * (B/C) = XS –Example: –3 jobs per second and each job needs 0.1 secs –U = 0.3 (30%)

6. 10/12/1999(c) Ian Davis6 Job flow balance assumption Total arrivals = Total Completions Reasonable since (A-C)/C  0 as T   λ = X U = λS (steady state limit theorem)

7. 10/12/1999(c) Ian Davis7 Generalizing to network Let there be n  1 services (servers) Arrival rate λ i = A i /T (at server i) Departure rate X i = C i /T (from server i) Server utilization U i = B i /T Mean service time S i = B i /C i Routing frequency (C ik task goes i  k) –q ik = C ik / C i if i = 1..n –q 0k = A 0k / A 0 if i = 0

8. 10/12/1999(c) Ian Davis8 Observations C i =  k=0..n C ik (i=1..n) C 0 =  k=0..n C k0 (completions from system) A 0 =  k=0..n A 0k (arrivals at system) X 0 =  k=1..n X i q i0 –(Output flow law)  k=0..n q ik =1

9. 10/12/1999(c) Ian Davis9 Queuing at device W i =  t=0..n queue lth(t) –ie. Total waiting time  i = W i / T –ie. Average queue length R i = W i / C i –ie. mean waiting time at I –Also called the response time  i = X i R i (Little’s law) –  i = W i / T = (C i / T) * (W i / C i )

10. 10/12/1999(c) Ian Davis10 Visit ratios Assuming X i =  k=0..n X k q ki –ie. balanced flow V i = X i / X o = C i / C o –(mean requests per task for i) –(mean completions at i per task completion) X i = V i X o (Forced flow law)

11. 10/12/1999(c) Ian Davis11 Visit ratio example Tasks generate on average 5 disk requests Disk throughput is 10 requests/second What is system throughput.. X o = X i /V i = 10(request/sec)/5(requests/job) ie. 2 jobs per second N.B. Using “measured” disk throughput N.B. All other measures irrelevant….

12. 10/12/1999(c) Ian Davis12 Interactive Response Time Suppose M users are using terminals Mean wait time per user at terminal R Mean think time per user Z Mean thinking and waiting time (R + Z) (R + Z) X o = M = mean number of users –(by Little’s law) R = (M/ X o - Z) –(Interactive Response Time Formula)

13. 10/12/1999(c) Ian Davis13 Interactive Response Time R = (  i=1..n  I )/X o = N/X o –(eg. Apply Little’s law to system) but  I /X o = V i R i since –  I = X i R i (Little’s law) –X i = V i X 0 (Forced flow law) Therefore R =  i=1..n V i R i –(General response time law)

14. 10/12/1999(c) Ian Davis14 Example 1 Parameters: –Each task generates 20 disk request –Disk utilization is 50% –Mean service time at disk 25 millisecs –25 terminals (think time is 18 seconds) X o = X i /V i = (U i /S i )/V i = (0.5/0.025)/20 –therefore X o = 1 job/second R = (M/X o ) - Z –therefore R = (25/1) - 18 = 7 seconds

15. 10/12/1999(c) Ian Davis15 Example 2 Parameters: –40 terminals (think time is 15 seconds) –Interactive response time is 5 seconds –Disk mean service time is 40 milliseconds –Each interactive task generates 10 disk I/O –Each batch job generates 5 disk I/O –Disk utilization is 90% Want to calculate throughput of batch and lower bound on interactive response time if batch throughput is tripled.

16. 10/12/1999(c) Ian Davis16 Example 2 continued Interactive throughput –I(X o ) = M/(Z+R) = 40/(15 + 5) = 2 jobs/second Disk i throughput {ie. interactive + batch } –X i = I(X i ) + B(X i ) = U i /S i = 0.9/0.040 = 22.5 requests/second {by utilization law} I(X i ) = I(V i ) I(X 0 ) = 10 * 2 = 20 req/sec – {by forced flow law} B(X i ) = X i - I(X i ) = 22.5 - 20 = 2.5 req/sec B(X 0 ) = B(X i ) / B(V i ) = 2.5/5 = 0.5 jobs/sec.

17. 10/12/1999(c) Ian Davis17 Example 2 continued Tripling B(X o ) = 1.5 batch jobs per second Disk i batch throughput –B(X i ) = B(V i ) * B(X o ) = 5 * 1.5 = 7.5 req/sec Maximum completion rate at disk –1/S i = 25 requests/second –I(X i )  25 - 7.5 = 17.5 requests/second I(X 0 ) = I(X i ) / I(V i )  17.5/10= 1.75 task/sec I(R) = (M/I(X 0 )) - Z  (40/1.75)-15 = 7.9sec –Assuming M,Z,V i & S i unchanged

18. 10/12/1999(c) Ian Davis18 Example 3 System A 16 terminals 25 disk I/O per job 80% disk utilization service time 0.042sec Think time 15 secs System B 10 terminals 16 disk I/O per job 40% disk utilization service time 0.042sec Think time 15 secs What are the response times for the current systems? What if A & B’s terminals and software are moved to a consolidate system C which uses the same disk drive?

19. 10/12/1999(c) Ian Davis19 Example 3 throughputs Since U i =X i S i & X i =V i X o => X o =U i / V i S i A(X o ) =.8/(25 * 0.042)  0.77 jobs/sec B(X o ) =.4/(16 * 0.042)  0.60 jobs/sec Why is B(X o ) < A(X o ) if B(V i ) < A (V i )? Because utilization lower, but why is this?

20. 10/12/1999(c) Ian Davis20 Why is utilization 40% Is a higher throughput possible? Minimum R is V i S i = 16 * 0.042 =.672 sec Max B(X o ) = M/(Z+R) = 10/(15+.672) = 0.638 > 0.6 jobs per sec What is the highest B(U i ) possible? U i = X o V i S i = 0.638*16*0.042 = 42.8% So in essence we can’t do enough work at 10 terminals to keep the drive busy.

21. 10/12/1999(c) Ian Davis21 Example 3 response times Since R = (M/X 0 ) - Z A(R) = (16/0.77) - 15 = 5.779 sec B(R) = (10/0.60) - 15 = 1.666 sec Fraction of I/O performed by A –A(X i )T / (A(X i ) + B(X i ))T = A(X i )/ (A(X i ) + B(X i )) = A(U i )/ (A(U i ) + B(U i )) = 80/120 = 2/3 –since X i = U i /S i & A(S i ) = B(S i )

22. 10/12/1999(c) Ian Davis22 Example 3 consolidate machine A(U i ) + B(U i ) = 120% > 100% Can assume C(U i )  100% Therefore C(X i )  1/S i = 1/.042 = 24 IO/sec A(X i ) = 24 * (2/3) = 16 IO/sec B(X i ) = 24 * (1/3) = 8 IO/sec A(X 0 ) = A(X i )/A(V i ) = 16/25 =.64 jobs/sec B(X 0 ) = B(X i )/B(V i ) = 8/16 =.5 jobs/secs

23. 10/12/1999(c) Ian Davis23 Example 3 response times Since R = (M/X 0 ) - Z A(R) = (16/.64) - 15 = 10 secs B(R) = (10/.5) - 15 = 5 secs N.B. M is not 16+10 because looking at the throughputs A(X 0 ) and B(X 0 ) separately.

24. 10/12/1999(c) Ian Davis24 The useful equations Utilization law (Very useful) –U i = X i S i { U i = λ i S i if assume λ i = X i } Little’s law –  i = X i R i Forced flow law (Very useful) –X i = V i X o General response time law –R =  i=1..n V i R i Interactive response time law (Very useful) –R = (M/X 0 ) - Z

25. 10/12/1999(c) Ian Davis25 Bottleneck analysis Let N be the number of tasks/jobs running what happens as N increases Assuming V i & S i remain unchanged then: –X i /X k =V i /V k and U i /U k =V i S i /V k S k unchanged Device i is saturated if U i = 100% Since U i /U k constant as N  U i increase by the same fraction when N  N+1 The device with largest U i (or equivalently the largest V i S i ) will be the first to saturate.

26. 10/12/1999(c) Ian Davis26 Bottleneck response time Response time obviously a minimum when only one task. In this case R o =  i V i S i System saturation N * occurs when N forces some task to be queued at some device. N * = R o /V b S b for any device such that U b =1 M = N * + Z/ V b S b will saturate the system.

27. 10/12/1999(c) Ian Davis27 Markov models System viewed as a set of states S 0 …S n, S -1 S 0 is start state, and S -1 final state Probability p ik transition between S i and S k Probabilities constant for lifetime of model  i=-1..n p ik = 1 p -1-1 = 1

28. 10/12/1999(c) Ian Davis28 Markov modelling goals Determine probability of entering S i Determine expected number of transitions before arriving in state S i Flag certain state transitions as significant Counting achieved by multiplying state transition probability by dummy variables

29. 10/12/1999(c) Ian Davis29 Markov modelling reduction For all i,k reduce multiple transitions S i  S k –Sum the probabilities of each such transition Select S i : S i does not move directly to S i Eliminate S i from the model using: –For all h,k : S h  S i  S k add S h  S k with p hi* p ik When all states loop eliminate p ii if i  -1 –For all k  i p ik = p ik * 1/(1-G) where G generating function describing p ii Terminate when have G for S 0  S -1

30. 10/12/1999(c) Ian Davis30 Use of generating function Set all dummy variables of no interest to 1 Probability that variable x visited ‘n’ times: –Expand G as polynomial in x –Coefficient associated with X n is probability Probability at most ‘n’ is sum of first n+1 coefficients etc.

31. 10/12/1999(c) Ian Davis31 Markov example 1 Software accesses a disk drive and a tape –accesses the disk drive S 1 with probability.2 –accesses the tape drive S 2 with probability.1 –Access pattern is independent of history Markov model: –S 0  S 1 p 01 =.2 S 0  S 2 p 02 =.1 S 0  S -1 p 0-1 =.7 –S 1  S 1 p 11 =.2 S 1  S 2 p 12 =.1 S 1  S -1 p 1-1 =.7 –S 2  S 1 p 21 =.2 S 2  S 2 p 22 =.1 S 2  S -1 p 2-1 =.7

32. 10/12/1999(c) Ian Davis32 Add in dummy variable Let d count disk accesses and t tape accesses Markov model: –S 0  S 1 p 01 =.2d S 0  S 2 p 02 =.1t S 0  S -1 p 0-1 =.7 –S 1  S 1 p 11 =.2d S 1  S 2 p 12 =.1t S 1  S -1 p 1-1 =.7 –S 2  S 1 p 21 =.2d S 2  S 2 p 22 =.1t S 2  S -1 p 2-1 =.7 Eliminating S 1  S 1 : –S 0  S 1 p 01 =.2d S 0  S 2 p 02 =.1t S 0  S -1 p 0-1 =.7 –S 1  S 2 p 12 =.1t(1/1-.2d) S 1  S -1 p 1-1 =.7(1/1-.2d) –S 2  S 1 p 11 =.2d(1/1-.1d) S 2  S -1 p 2-1 =.7(1/1-.1d)

33. 10/12/1999(c) Ian Davis33 Eliminating S 1 : –S 0  S 2 p 01 *p 12 + p 02 =.2d *.1t(1/1-.2d) +.1t –S 0  S -1 p 0-1 =.7 –S 2  S 2 p 21 *p 12 =.2d(1/1-.1d) *.1t(1/1-.2d) –S 2  S -1 p 2-1 =.7(1/1-.1d) Eliminating S 2  S 2 : –S 2  S -1 p 2-1 = (1/1 -.2d(1/1-.1d) *.1t(1/1-.2d)) *.7(1/1-.1d) Eliminating S 2 computing S 0  S -1 G: –p 02 p 2-1 + p 0-1 = (.2d*.1t(1/1-.2d)+.1t) * (1/1 -.2d(1/1-.1d) *.1t(1/1-.2d)) *.7(1/1-.1d) +.7 – S 0  S 2 p 02 =.1t S 0  S -1 p 0-1 =.7 –S 1  S 2 p 12 =.1t(1/1-.2d) S 1  S -1 p 1-1 =.7(1/1-.2d) –S 2  S 1 p 11 =.2d(1/1-.1d) S 2  S -1 p 2-1 =.7(1/1-.1d)

34. 10/12/1999(c) Ian Davis34 Probability of ‘n’ disk accesses t:=1; taylor(G, d);.7777777778 +.1728395062 d +.03840877915 d 2 +.008535284255 d 3 +.001896729834 d 4 +.0004214955188 d 5 + O(d 6 ) Truncate and set d=1 for  ‘n’ accesses Could parameterize 0.2=p and 0.1=q and 0.7=(1-p-q)

35. 10/12/1999(c) Ian Davis35 Average number of disk accesses G’ = diff(G) w.r.t variable d: –0*p(miss)*d 0 +1*p(once)*d 1 +2*p(twice)*d 2 … V(accesses) = E(accesses 2 ) - E 2 (accesses) –G’’ = diff(G’) = E(accesses 2 ) - E(accesses) Now set d := 1 –G’ = 0.2857142857 (Average # disk accesses) –(G’’ + G’) - G’*G’ = 0.3673469388 (Variance)