1. Little’s Law & Operational Laws

2. Little’s Law
Proportionality relation between the average number of jobs (E[N]) in a system and the average system time (E[T]) of those jobs (the job arrival rate is the proportionality constant)
No assumptions about the arrival or service processes
Two similar versions: Open and closed systems
Little’s Law for an ergodic open system: E[N] = λE[T]
λ is the average job arrival rate in the system
Little’s law for an ergodic closed system: N = XE[T]
N is constant and equal to the multi-programming level
X is the system throughput (rate of job completion)
E[T] = E[R] + E[Z], where R is response time, and Z think time
Alternative version (Response Time Law): E[R] = N/X – E[Z]

3. Applying Little’s Law – (1)1   2
Assume that the above system is ergodic with X =  = 1+2 , where 1 (2) is he arrival rate of red (green) jobs
Ergodicity requires  < 
Utilization Law: system = server
Let  (i) be fraction of time server is busy (with type i jobs) – Note: This is also the average number of (type i) jobs in service
Little’s Law implies  = / (i = i/), or  = E[S]
Note: If E[S1]  E[S2], then i = i/i, where i = E[Si]
Time in queue: system = queue
E[NQi] = i E[TQi]

4. Applying Little’s Law – (2)… N in
out
Assume N = 10, E[Z] = 5secs, and E[R] = 15secs
What is the system throughput X?
N = X E[T] = X(E[Z] + E[R])
 X = N/(E[R] + E[Z]) = 10/(5+15) = 0.5 jobs/sec

5. Forced Flow Law
Relates system throughput to throughput of individual device
X is system throughput, Xi is throughput of device i
Vi is average number of visit to device i per job
Forced Flow Law: Xi = E[Vi]X
Basically accounts for the fact that for every system completion, there must have been E[Vi] completions (visits) at device i, so the rate of completion at device i must be higher in that same proportion

6. Bottleneck Law
Di is the per job service demand at device i, i.e., total demand across all visits to device i
Di = Si,1 + Si,2 + … + Si,Vi
This implies E[Di] = E[Vi]E[Si], if Vi and Si are independent (a typical scenario)
Note that E[Di] = Bi/C, where Bi is the busy time at device i and C is the number of service completion during that time period
Bottleneck Law: i = XE[Di]
(i = XiE[Si] = XE[Vi]E[Si] = X E[Di])
Rate of outside arrivals is X and each such arrival contributes E[Di] worth of service time at device i, so that device i is busy a fraction of time (i) equal to X E[Di]

7. Summary Open systems Closed systems E[N] = λE[T] (Little’s Law)
E[Nsubsystem] = subsytem E[Tsubsystem] (Little’s Law)
Closed systems
N = XE[T] = X(E[R] + E[Z]) (Little’s Law)
E[R] = N/X – E[Z] (Response Time Law)
i = i/i = i E[Si] = Xi E[Si] (Utilization Law)
Xi = E[Vi]X (Forced Flow Law)
i = XE[Di] (Bottleneck Law)

8. Building on Little’s Law
Bounds on throughput and response time for closed systems
where Dmax = maxi{E[Di]} (system bottleneck)
First expression holds for N small, while second expression holds for N large
N* as the N value for which the two expressions are equal, i.e., N* = (D+E[Z])/Dmax
When N > N*, throughput is dominated by bottleneck component

9. Example (1) … System A: DCPU = 4.6 and Ddisk = 4.0
N = 10, E[Z] =5
CPU
System A: DCPU = 4.6 and Ddisk = 4.0
System B: DCPU = 4.9 and Ddisk = 1.9
Which system has higher throughput?
First compute N* (N* = (D+E[Z])/Dmax) for both systems
DA = DCPU + Ddisk = 8.6, DA,max = 4.6  NA* = (8.6+5)/4.6 = 2.95
DB = DCPU + Ddisk = 6.8, DB,max = 4.9  NB* = (6.8+5)/4.9 = 2.41
NA* & NB* << N
So in both cases, the throughput is dominated by Dmax, and System A wins because it has a lower Dmax value

10. Summary
N = XE[T] = X(E[R] + E[Z])
E[R] = N/X – E[Z] (Response Time Law)
i = i/i = i E[Si] = Xi E[Si] (Utilization Law)
Xi = E[Vi]X (Forced Flow Law)
i = XE[Di] (Bottleneck Law)
Example (2)
Measurements for interactive system with N = 20 and E[Z] = 15 secs:
T = 650 secs (duration of measurements)
BCPU = 400 secs
Bslowdisk = 100 secs
Bfastdisk = 600 secs
C = CCPU = 200 jobs
Cslowdisk = 2,000 jobs
Cfastdisk = 20,000 jobs
Improvements under consideration (are they worth it?)
Faster CPU
Rebalancing disks
Add a second fast disk, and split the load of the original fast disk using it (Note: This is sub-optimal – should rebalance across all three disks)
Combine all three improvements, including rebalancing all three disks

11. Example (2) Original system
Summary
N = XE[T] = X(E[R] + E[Z])
E[R] = N/X – E[Z] (Response Time Law)
i = i/i = i E[Si] = Xi E[Si] (Utilization Law)
Xi = E[Vi]X (Forced Flow Law)
i = XE[Di] (Bottleneck Law)
Example (2) Original system
Measurements for interactive system with N = 20 and E[Z] = 15 secs:
T = 650 secs (duration of measurements)
BCPU = 400 secs, Bslowdisk = 100 secs, Bfastdisk = 600 secs
C = CCPU = 200 jobs. Cslowdisk = 2,000 jobs, Cfastdisk = 20,000 jobs
Intermediate quantities
E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job
E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0.5 secs/job
E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job
E[VCPU] = CCPU/C = 200 visits/200 jobs = 1.0 visit/job
E[Vslowdisk] = Cslowdisk/C = 2,000 visits/200 jobs = 10 visits/job
E[Vfastdisk] = Cfastdisk/C = 20,000 visits/200 jobs = 100 visits/job
E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2.0 secs/visit
E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2,000 visits = 0.05 sec/visit
E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20,000 visits =0.03 sec/visit
Original system: Dmax= 3 secs/job, D = 5.5 secs/job, N* = 20.5/3  7 << N = 20
So X  1/Dmax= 0.33 jobs/sec and E[R]  NDmax – E[Z] = 45 secs

12. Example (2a) – Faster CPU
Summary
N = XE[T] = X(E[R] + E[Z])
E[R] = N/X – E[Z] (Response Time Law)
i = i/i = i E[Si] = Xi E[Si] (Utilization Law)
Xi = E[Vi]X (Forced Flow Law)
i = XE[Di] (Bottleneck Law)
Example (2a) – Faster CPU
Metrics for interactive system with N = 20 and E[Z] = 15 secs:
E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job
E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0.5 secs/job
E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job (bottleneck)
E[VCPU] = CCPU/C = 200 visits/200 jobs = 1.0 visit/job
E[Vslowdisk] = Cslowdisk/C = 2,000 visits/200 jobs = 10 visits/job
E[Vfastdisk] = Cfastdisk/C = 20,000 visits/200 jobs = 100 visits/job
E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2.0 secs/visit
E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2,000 visits = 0.05 sec/visit
E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20,000 visits =0.03 sec/visit
Faster CPU (twice as fast): E[DCPU] = 1.0 sec/job, but Dmax remains unchanged and N* stays approximately constant
Hardly any improvement as the fast disk is the bottleneck

13. Example (2b) – Rebalance Disks
Summary
N = XE[T] = X(E[R] + E[Z])
E[R] = N/X – E[Z] (Response Time Law)
i = i/i = i E[Si] = Xi E[Si] (Utilization Law)
Xi = E[Vi]X (Forced Flow Law)
i = XE[Di] (Bottleneck Law)
Example (2b) – Rebalance Disks
Metrics for interactive system with N = 20 and E[Z] = 15 secs:
E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job
E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0.5 secs/job
E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job
E[VCPU] = CCPU/C = 200 visits/200 jobs = 1.0 visit/job
E[Vslowdisk] = Cslowdisk/C = 2,000 visits/200 jobs = 10 visits/job
E[Vfastdisk] = Cfastdisk/C = 20,000 visits/200 jobs = 100 visits/job
E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2.0 secs/visit
E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2,000 visits = 0.05 sec/visit
E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20,000 visits =0.03 sec/visit
Clearly, we have overloaded the fast disk. Optimal balancing is such that
E[Vslowdisk]  E[Sslowdisk] = E[Vfastdisk]  E[Sfastdisk] or E[Vslowdisk]  0.05 = E[Vfastdisk]  0.03
while keeping E[Vslowdisk] + E[Vfastdisk] = 110
This gives E[Vfastdisk]  69 and E[Vslowdisk]  41 and consequently E[Dslowdisk] = E[Dfastdisk] = 2.06 secs, and therefore a new value of Dmax = 2.06 secs
D is now 6.12 secs and N* becomes 10, which remains smaller than N. Hence, throughput and response time are still dominated by Dmax, and the system improves to X  jobs/sec and E[R] = 26.2 secs

14. Example (2c) – Add Another Fast Disk
Summary
N = XE[T] = X(E[R] + E[Z])
E[R] = N/X – E[Z] (Response Time Law)
i = i/i = i E[Si] = Xi E[Si] (Utilization Law)
Xi = E[Vi]X (Forced Flow Law)
i = XE[Di] (Bottleneck Law)
Example (2c) – Add Another Fast Disk
Metrics for interactive system with N = 20 and E[Z] = 15 secs:
E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job
E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0.5 secs/job
E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job
E[VCPU] = CCPU/C = 200 visits/200 jobs = 1.0 visit/job
E[Vslowdisk] = Cslowdisk/C = 2,000 visits/200 jobs = 10 visits/job
E[Vfastdisk] = Cfastdisk/C = 20,000 visits/200 jobs = 100 visits/job
E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2.0 secs/visit
E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2,000 visits = 0.05 sec/visit
E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20,000 visits =0.03 sec/visit
If we split the fast disk load across two fast disks, we get
E[Dfastdisk1] = E[Dfastdisk2] = 1.5 jobs/sec
The slow disk becomes the bottleneck which yieds a new value of Dmax = 2.0 secs
D does not change but since Dmax does, N* = (D+E[Z])/Dmax  10. Still smaller than N, so that the system improves to X = 0.5 jobs/sec and E[R] = 25 secs

15. Example (2d) – Combine All Improvements
Summary
N = XE[T] = X(E[R] + E[Z])
E[R] = N/X – E[Z] (Response Time Law)
i = i/i = i E[Si] = Xi E[Si] (Utilization Law)
Xi = E[Vi]X (Forced Flow Law)
i = XE[Di] (Bottleneck Law)
Example (2d) – Combine All Improvements
Metrics for interactive system with N = 20 and E[Z] = 15 secs:
E[DCPU] = BCPU/C = 400 secs/200 jobs = 2.0 secs/job
E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0.5 secs/job
E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3.0 secs/job
E[VCPU] = CCPU/C = 200 visits/200 jobs = 1.0 visit/job
E[Vslowdisk] = Cslowdisk/C = 2,000 visits/200 jobs = 10 visits/job
E[Vfastdisk] = Cfastdisk/C = 20,000 visits/200 jobs = 100 visits/job
E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2.0 secs/visit
E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2,000 visits = 0.05 sec/visit
E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20,000 visits =0.03 sec/visit
With faster CPU and two fast disks and rebalancing load across three disks, we get
E[DCPU] = 1 sec/job, E[Dfastdisk1] = E[Dfastdisk2] = E[Dslowdisk] = 1.27 jobs/sec  D = 4.8 secs
Where we have used E[Vslowdisk]  0.05 = E[Vfastdisk]  0.03, and E[Vslowdisk] + 2E[Vfastdisk] = 110
This gives Dmax = 1.27 secs and N* = (D+E[Z])/Dmax  16 that remains smaller than N
Hence, throughput and response time are still dominated by Dmax, and the system improves to X  jobs/sec and E[R] = 10.4 secs