1. 1 Discrete Mathematics Summer 2004 By Dan Barrish-Flood originally for Fundamental Algorithms For use by Harper Langston in D.M.

2. 2 A problem is a precise specification of input and desired output An algorithm is a precise specification of a method for solving a problem. What is an Algorithm?

3. 3 What We Want in an Algorithm Correctness –Always halts –Always gives the right answer Efficiency –Time –Space Simplicity

4. 4 A Typical Problem Sorting Input: A sequence b 1, b 2,... b n Output: A permutation of that sequence b 1, b 2,... b n such that

5. 5 A Typical Algorithm

6. 6 Running Time t J is the number of times the “while” loop test in line 5 is executed for that value of j. Total running time is c 1 (n) + c 2 (n-1) +…+c 8 (n-1)

7. 7 Trying to determine the exact running time is tedious, complicated and platform-specific. Instead, we’ll bound the worst-case running time to within a constant factor for large inputs. Why worst-case? –It’s easier. –It’s a guarantee. –It’s well-defined. –Average case is often no better. A Better Way to Estimate Running Time

8. 8 Insertion Sort, Redux Outer loop happens about n times. Inner loop is done j times for each j, worst case. j is always < n So running time is upper-bounded by about  “Insertion sort runs in  (n 2 ) time, worst-case.”

9. 9 We’ll make all the “about”s and “rough”s precise. These so-called asymptotic running times dominate constant factors for large enough inputs.

10. 10 Functions constants polynomials exponentials logarithms and polylogarithms factorial

11. 11 Polynomials

12. 12 f(n) is monotonically increasing iff I.e. It never “goes down” — always increases or stays the same. For strictly increasing, replace  in the above with <. I.e. The function is always going up.

13. 13 Are constant functions monotonically increasing? Yes! (But not strictly increasing.) If, then is strictly increasing. So even increases forever to

14. 14 Exponentials

15. 15 Logarithms

16. 16 Log Facts 1.n = b log b n (definition) 2.log b (xy)=log b x + log b y 3.log b a n = nlog b a 4.log b x = log c x / log c b 5.log b (1/a) = -log b a 6.log b a = 1/(log a b) 7.a log b c = c log b a if b > 1, then log b n is strictly increasing.

17. 17 More About Logs A polylogarithm is just

18. 18 Factorial

19. 19 More About Monotonically Increasing Functions

20. 20 Bounding Functions We are more interested in finding upper and lower bounds on functions that describe the running times of algorithms, rather than finding the exact function. Also, we are only interested in considering large inputs: everything is fast for small inputs. So we want asymptotic bounds. Finally, we are willing to neglect constant factors when bounding. They are tedious to compute, and are often hardware-dependent anyway.

21. 21 Example

22. 22 Notation for Function Bounds

23. 23 Big-Oh Facts It denotes an upper bound, not necessarily a tight bound. Thus, The “=” is misleading. E.g is meaningless. Transitivity:

24. 24 Examples We’ve seen that 3n + 8 = O(n). Is 10n = O(3n + 8)? Yes: choose c = 10, n 0 = 1: 10n  n  + 80

25. 25 More Examples 2n 2 + 5n - 8 = O(n 2 )? Yes: 2n 2 + 5n - 8  cn 2  n 2 : 2 + 5/n - 8/n 2  c Choose c = 3; subtracting 2: 5/n - 8/n 2  Choose n o = 5

26. 26 More Examples

27. 27 Lower Bound Notation

28. 28 “Tight” Bound Notation

29. 29 More Facts

30. 30 Examples

31. 31 Another Upper-Bound Notation

32. 32 Functions and Bounds

33. 33 Logarithms

34. 34 Polynomials

35. 35 Comparing Functions