1. Introduction to Analysing Costs
COMP 103
Introduction to Analysing Costs
Marcus Frean, Rashina Hoda, and Peter Andreae
Thomas Kuehne
School of Engineering and Computer Science, Victoria University of Wellington
2016-T2 Lecture 08

2. RECAP-TODAY RECAP Implementing the fundamental ArrayList methods2
RECAP
Implementing the fundamental ArrayList methods
AbstractList  ArrayList does not have to implement everything
TODAY
introduction to analysing cost

3. Analysing Costs (in general)3
How can we determine the costs of a program?
Time:
Run the program and count the milliseconds/minutes/days.
Count number of steps/operations the algorithm will take.
Space:
Measure the amount of memory the program occupies.
Count the number of elementary data items the algorithm stores.
Applies to Programs or Algorithms? Both.
programs: called “benchmarking”
algorithms: called “analysis”

4. Benchmarking: program cost4
Measure:
actual programs, on real machines, with specific input
measure elapsed time
System.currentTimeMillis () → time from the system clock in milliseconds
measure real memory
Problems:
what input? ⇒ use large data sets don’t include user input
other users/processes? ⇒ minimise average over many runs
which computer? ⇒ specify details
how to compare cross-platform? ⇒ measure cost at an abstract level

5. Analysis: Algorithm “complexity”5
Abstract away from the details of
the hardware, the operating system
the programming language, the compiler
the specific input
Measure number of “steps” as a function of the data size
best case (easy, but not interesting)
worst case (usually easy)
average case (harder)
The precise number of steps is not required
3.47 n2 - 67n steps
3n log(n) + 5n - 3 steps
Rather, we are interested in the complexity class

6. Analysis: The Big Picture6
cost
f1(n)
f1(n) = n n + 2
3,000,000
f2(n) = n
2,000,000
1,000,000
f2(n)
input size
125
250
500
1000
2000
Only care about with what shape cost grows with input size

7. Analysis: Notation Big-O Notation7
Big-O Notation
f(n) is O(g(n)), if there are two positive constants, s and n0 with f(n) £ s  | g(n) |, for all n ³ n0
cost
s  g(n)
3000
2000
f(n)
1000
input size
125
250
500 n0
1000
2000

8. Big-O Notation 8
Only care for large input sets and ignore constant factors
3.47 n n steps  O(n2)
3n log n + 12n steps  O(n log n)
Lower-order terms become insignificant for large n
Multiplication factors don’t tell us how things “scale up”
We care about how cost grows with input size

9. Big-O classes 9
“Asymptotic cost”, or “big-O” cost describes how cost grows with input size
Examples:
O(1) constant cost is independent of n : Fixed cost!
O(log n) logarithmic cost grows up by 1, every time n doubles : Slow growth!
O(n) linear cost grows linearly with n : can often be beaten
O(n2) quadratic costs grows like n  n : severely limits problem size

10. Big-O costs Cost input size 10 N O(1) Constant O(log N) Logarithmic
Linear
O(N log N)
O(N^2)
Quadratic
O(N^3)
Cubic
O(N!)
Factorial 1 5
0.00
2.32
11.61 25
125
120 10
3.32
33.22
100
1000 15
3.91
58.60
225
3375
1.31E+12 20
4.32
86.44
400
8000
2.43E+18
O(n3)
O(n2)
Cost
O(n log n)
O(n)
O(log n)
O(1)
input size

11. Manageable Problem Sizes11 N
1 min
1 h
1 day
1 week
1 year n
6 ´ 107
3.6 ´ 109
8.64 ´ 1010
6.05 ´ 1011
3.15´ 1013
n log n
2.8 ´ 106
1.3 ´ 108
2.75 ´ 109
1.77 ´ 1010
7.97 ´ 1011 n2
7.75 ´ 103
6.0 ´ 104
2.94 ´ 105
7.78 ´ 105
5.62 ´ 106 n3
3.91 ´ 102
1.53 ´ 13
4.42 ´ 103
8.46 ´ 103
3.16 ´ 104 2n 25 31 36 39 44
High asymptotic cost severely limits problem size

12. What is a “step”? Of importance comparing data moving data12
Of importance
comparing data
moving data
anything you consider to be “expensive”
public E remove (int index){     if (index < 0 || index >= count) throw new ….Exception();     E ans = data[index];     for (int i=index+1; i< count; i++)       data[i-1]=data[i];     
   count--; data[count] = null;     return ans;   }
← Key Step

13. ArrayList: get, set, remove13
Assume an ArrayList contains n items.
Cost of get and set:
best, worst, average:
Cost of Remove:
worst case:
what is the worst case?
how many steps?
average case:
what is the average case? n

14. ArrayList: add (at some position)14
public void add(int index, E item){ if (index<0 || index>count) throw new IndexOutOfBoundsException();
ensureCapacity();
for (int i=count; i > index; i--)       data[i]=data[i-1];
data[index]=item; count++;
}  
Cost of add(index, value):
key steps? n

15. ArrayList: add at end Cost of add(value): which are the key steps?15
Cost of add(value):
worst case:
average case:
public void add (E item) {     ensureCapacity();     data[count++] = item; } private void ensureCapacity () {     if (count < data.length) return;     E [ ] newArray = (E[ ]) (new Object[data.length * 2]);     for (int i = 0; i < count; i++)       newArray[i] = data[i];     data = newArray; }  n
which are the key steps?

16. ArrayList: Amortised cost16
Some single “adds” are expensive, but what is the big picture?
“Amortised” cost: total cost of adding n items, divided by n:
first 10: cost = 1 each total = 10
11th: cost = 10+1 total = 21
12-20: cost = 1 each total = 30
21st: cost = 20+1 total = 51
22-40: cost = 1 each total = 70
41st: cost = 40+1 total = 111
42-80: cost = 1 each total = 150 :
- n total =
Amortised cost ( ) = ?
per item

17. ArrayList: Amortised cost17
Starting with array length = 1 makes it easier to see the pattern
first : cost = 1 each total = 1
2nd: cost = 1+1 total = 3
3rd: cost = total = 6
4th: cost = 1 each total = 7
5th: cost = total = 12
6-8: cost = 1 each total = 15
9th: cost = total = 24
10-16: cost = 1 each total = 31 :
n total =
Amortised cost ( ) =
look at pattern to right (1,3,7,15,31) -> 2^x -1 OR calculate sum i=0..k (2^i) = 2^(k+1)-1
how many copy steps are there? n = 2^steps = 2^k => k=log_2 n
now insert steps into count formular
steps = 2^(log2(n)+1)-1 -> n
-> like previous example with a factor of ten
finally: divide by n to achieve amortised cost.
per item

18. ArrayList costs: Summary18
get O(1)
set O(1)
remove O(n)
add (at i) O(n) (worst and average)
add (at end) O(1) (average)
O(n) (worst)
O(1) (amortised average)
To think about:
What would the amortised cost be if the array size were increased by a fixed amount (say 10) each time?