1. CSE 326 Asymptotic Analysis David Kaplan Dept of Computer Science & Engineering Autumn 2001

2. Asymptotic AnalysisCSE 326 Autumn 20012 Housekeeping  Homework 1 status  Join cse326@cs  Poll:  Slide format(s)?  Office hours today?

3. Asymptotic AnalysisCSE 326 Autumn 20013 Analyzing Algorithms Analyze algorithms to gauge:  Time complexity (running time)  Space complexity (memory use) Input size is indicated by a number n  sometimes have multiple inputs, e.g. m and n Running time is a function of n n, n 2, n log n, 18 + 3n(log n 2 ) + 5n 3

4. Asymptotic AnalysisCSE 326 Autumn 20014 RAM Model RAM (random access machine)  Ideal single-processor machine (serialized operations)  “Standard” instruction set (load, add, store, etc.)  All operations take 1 time unit (including, for our purposes, each C++ statement)

5. Asymptotic AnalysisCSE 326 Autumn 20015 Order Notation (aka Big-O) Big-O T ( n ) = O( f ( n ) ) Exist positive constants c, n 0 such that T(n)  cf(n) for all n  n 0 Upper bound Omega T(n) =  ( f ( n ) ) Exist positive constants c, n 0 such that T(n)  cf(n) for all n  n 0 Lower bound Theta T ( n ) = θ( f ( n ) ) T(n) = O(f(n)) AND T(n) =  (f(n)) Tight bound little-o T ( n ) = o ( f ( n ) ) T(n) = O(f(n)) AND T(n) != θ(f(n)) Strict upper bound

6. Asymptotic AnalysisCSE 326 Autumn 20016 Simplifying with Big-O By definition, Big-O allows us to: Eliminate low order terms  4n + 5  4n  0.5 n log n - 2n + 7  0.5 n log n Eliminate constant coefficients  4n  n  0.5 n log n  n log n  log n 2 = 2 log n  log n  log 3 n = (log 3 2) log n  log n But when might constants or low-order terms matter?

7. Asymptotic AnalysisCSE 326 Autumn 20017 Big-O Examples n 2 + 100 n = O(n 2 ) follows from … ( n 2 + 100 n )  2 n 2 for n  10 n 2 + 100 n =  (n 2 ) follows from …( n 2 + 100 n )  1 n 2 for n  0 n 2 + 100 n =  (n 2 ) by definition n log n = O(n 2 ) n log n =  (n log n) n log n =  (n)

8. Asymptotic AnalysisCSE 326 Autumn 20018 Big-O Usage Order notation is not symmetric:  we can say2n 2 + 4n = O(n 2 )  … but neverO(n 2 ) = 2n 2 + 4n Right-hand side is a crudification of the left Order expressions on left can produce unusual- looking, but true, statements: O(n 2 ) = O(n 3 )  (n 3 ) =  (n 2 )

9. Asymptotic AnalysisCSE 326 Autumn 20019 Big-O Comparisons Function A n 3 + 2n 2 n 0.1 n + 100n 0.1 5n 5 n -15 2 n /100 8 2log n Function #2 100n 2 + 1000 log n 2n + 10 log n n! 1000n 15 3n 7 + 7n vs.

10. Asymptotic AnalysisCSE 326 Autumn 200110 Race 1 100n 2 + 1000 vs. n 3 + 2n 2

11. Asymptotic AnalysisCSE 326 Autumn 200111 Race 2 n 0.1 log n vs. In this one, crossover point is very late! So, which algorithm is really better???

12. Asymptotic AnalysisCSE 326 Autumn 200112 Race C n + 100n 0.1 2n + 10 log n vs. Is the “better” algorithm asymptotically better???

13. Asymptotic AnalysisCSE 326 Autumn 200113 Race 4 5n 5 n! vs.

14. Asymptotic AnalysisCSE 326 Autumn 200114 Race 5 n -15 2 n /1001000n 15 vs.

15. Asymptotic AnalysisCSE 326 Autumn 200115 Race VI 8 2log(n) 3n 7 + 7n vs.

16. Asymptotic AnalysisCSE 326 Autumn 200116 Big-O Winners (i.e. losers) Function A n 3 + 2n 2 n 0.1 n + 100n 0.1 5n 5 n -15 2 n /100 8 2log n Function #2 100n 2 + 1000 log n 2n + 10 log n n! 1000n 15 3n 7 + 7n vs. Winner O(n 2 ) O(log n) O(n) TIE O(n 5 ) O(n 15 ) O(n 6 ) why???

17. Asymptotic AnalysisCSE 326 Autumn 200117 Big-O Common Names constant:O(1) logarithmic:O(log n) linear:O(n) log-linear:O(n log n) superlinear:O(n 1+c )(c is a constant > 0) quadratic:O(n 2 ) polynomial:O(n k )(k is a constant) exponential:O(c n )(c is a constant > 1)

18. Asymptotic AnalysisCSE 326 Autumn 200118 Kinds of Analysis Running time may depend on actual input, not just length of input Distinguish  Worst case  Your worst enemy is choosing input  Average case  Assume probability distribution of inputs  Amortized  Average time over many runs  Best case (not too useful)

19. Asymptotic AnalysisCSE 326 Autumn 200119 Analyzing Code C++ operations Consecutive stmts Conditionals Loops Function calls Recursive functions constant time sum of times larger branch plus test sum of iterations cost of function body solve recursive equation

20. Asymptotic AnalysisCSE 326 Autumn 200120 Nested Loops for i = 1 to n do for j = 1 to n do sum = sum + 1

21. Asymptotic AnalysisCSE 326 Autumn 200121 Dependent Nested Loops for i = 1 to n do for j = i to n do sum = sum + 1

22. Asymptotic AnalysisCSE 326 Autumn 200122 Recursion  A recursive procedure can often be analyzed by solving a recursive equation  Basic form: T(n) = base case: some constant recursive case: T(subproblems) + T(combine)  Result depends upon  how many subproblems  how much smaller are subproblems  how costly to combine solutions (coefficients)

23. Asymptotic AnalysisCSE 326 Autumn 200123 Sum of Queue SumQueue(Q) if (Q.length == 0 ) return 0 else return Q.dequeue() + SumQueue(Q) One subproblem Linear reduction in size (decrease by 1) Combining: constant (cost of 1 add) T(0)  b T(n)  c + T(n – 1) for n>0

24. Asymptotic AnalysisCSE 326 Autumn 200124 Sum of Queue Solution T(n)  c + c + T(n-2)  c + c + c + T(n-3)  kc + T(n-k) for all k  nc + T(0) for k=n  cn + b = O(n) T(0)  b T(n)  c + T(n – 1) for n>0 Equation: Solution:

25. Asymptotic AnalysisCSE 326 Autumn 200125 Binary Search BinarySearch(A, x) Search A, a sorted array, for item x One subproblem, half as large 7123035758387909799 T(1)  b T(n)  T(n/2) + c for n>1 Equation:

26. Asymptotic AnalysisCSE 326 Autumn 200126 Binary Search: Solution T(n)  T(n/2) + c  T(n/4) + c + c  T(n/8) + c + c + c  T(n/2 k ) + kc  T(1) + c log n where k = log n  b + c log n = O(log n) T(1)  b T(n)  T(n/2) + c for n>1 Equation: Solution: