A Introduction to Computing II Lecture 5: Complexity of Algorithms Fall Session 2000

How long does a program take to run? Depends on what the input is May just depend on some parameter(s) of the input Example: copying the String depends on the length “a”.clone( ) is less work than “abc…z”.clone()

To Quantify That... Assume some simple operations take fixed time e.g. a[i]= 0; => 1 time unit Complex operations depend on how many simpler operations are performed e.g. for i := 1 to n do a[i] = 0; => n time units

Do constants matter? Q. What if a[i] = 0 and x = 0 don’t take the same time? A. As it happens, this isn’t as important as loops, recursions etc. Therefore, we will use an asymptotic notation, which ignores the exact value of these constants...

The Big O( ) Definition: If there is a function f(n) where there exists N and c such that for any input of length n > N, the running time of a program P is bounded as Time(P) < c f(n) we say that P is O(f(n))

The Big O( ) What does it really mean? n Time f(x) n > N n < N N

The Big O( ) WARNING: CORRUPT NOTATION We write… g(n) = O( f(n) ) even though g(n) and O( f(n) ) are not equivalent.

Examples x = 1; for j := 1 to n do A[j] = 0; for j := 1 to n do for k := 1 to n do A[j][k] = 0; for (int j = n; j != 0; j /= 2) A[j] = 0 f(n): prints all strings of { a, b }* of length n O(1) constant O(n) linear O(n 2 ) quadratic O(log n) logarithmic O(e n ) exponential ExampleGrowth A.k.a.

Worst Case Analysis Big O( ) is worst-case in that the real running time may be much less ( f(n) is an upper-bound): Example: String s = … ; for (int j = 0; j < s.length( ); j++) if (s [ j ] = “a”) break;

Worst Case Analysis Big O( ) is worst-case in that the real running time may be much less ( f(n) is an upper-bound): Example: String s = … ; for (int j = 0; j < s.length( ); j++) if (s [ j ] = “a”) break; Time = O(n)

Best-case Analysis We may choose to analyze the least time the program could take to run This is called big-  notation If P is O( f(n) ) and  ( f(n) ) we say: P is  ( f(n) )

Intuitively... O,  and  do for functions what less than, equal and greater than do for numbers. f(x) = O ( g(x) ) f(x) =  ( g(x) ) f(x) =  ( g(x) ) (i < j) (i = j) (i > j)

f(x) = o ( g(x) ) f(x) =  ( g(x) ) A little more notation “Little-oh” “Little-omega” Lower-case letters act like the corresponding strict inequalities ( ), i.e. it is known that f(x) =  ( g(x) ):

Some Things to Note If P = O( 1 ), it is also true that P = O( n ) If P = O( n k ), it is also true that P = O( n j ) for (j > k) 1. O( ) is a bound, so: 2. If P = O( f(n) + g(n) ) and f(n) = O( g(n) ) then P = O( g(n) ) Example: P = O( x 2 + x ) => P = O( x 2 )

More examples: What about adding two numbers?? 1) In what parameter do we do the analysis? 2) O,  and  ?

More examples: What about adding two numbers?? Let n be the number of digits in the numbers (assume same length) a d a (d-1) a (d-2)... a i … a 3 a 2 a 1 b d b (d-1) b (d-2)... b i … b 3 b 2 b 1 c (d+1) c d c (d-1) c (d-2)... c i … c 3 c 2 c 1 +

More examples: What about adding two numbers?? We do exactly one (primitive) addition for each of d digits =>  ( d ) a d a (d-1) a (d-2)... a i … a 3 a 2 a 1 b d b (d-1) b (d-2)... b i … b 3 b 2 b 1 c (d+1) c d c (d-1) c (d-2)... c i … c 3 c 2 c 1 +

The parameter is important! Let’s say we did the analysis on the number itself rather than how many digits it contains… … is it still linear ???

The parameter is important! Let’s say we did the analysis on the number itself rather than how many digits it contains… … is it still linear ??? NO! If the number is n, d = log n O( d ) = O ( log n )

So what is O(1) in Java Primitive math operations (i.e. +,-,*, / on ints, doubles, etc) Accessing simple variables (and data members) Accessing an array, A[i]

So what is not O(1) in Java Method calls usually aren’t: depends on the body of the method This includes Java library calls like in java.lang.math Loops of any kind

Another example: Exponentiation What is the order of growth of and can we do better than: Function exp(m,n) ::= { result := 1 while (n > 0) result := result * m n := n -1 }

Another example: Exponentiation What is the order of growth of and can we do better than: Answer #1: O( n ) Answer #2: yes…

Better Exponentiation Observe: We can rewrite exponentiations like 5 13 = 5 (5 2 ) 2 (( 5 2 ) 2 ) 2 This has only seven multiplications (instead of thirteen)

Better Exponentiation Function exp(m, n) ::= { result := 1 while (n > 0) if ( n is even) m := m 2 n := n /2 else result := result * m n := n - 1 }

Order of Growth?? Best-case: We always divide by 2 until n := 1 =>  ( log n ) iterations Worst-case: If we’re forced into the other branch (n odd) it will be even next time, so : 2 log n = O( log n ) Conclusion:  ( log n )

Any questions?