1. 1 i206: Lecture 6: Math Review, Begin Analysis of Algorithms Marti Hearst Spring 2012

2. 2 Examples of Social Algorithms?

3. 3 Which Algorithm is Better? What do we mean by “ Better ” ? Sorting algorithms –Bubble sort –Insertion sort –Shell sort –Merge sort –Heapsort –Quicksort –Radix sort –… Search algorithms –Linear search –Binary search –Breadth first search –Depth first search –…

4. 4 Analysis of Algorithms Characterizing the running times of algorithms and data structure operations Secondarily, characterizing the space usage of algorithms and data structures

5. 5 Why Analysis of Algorithms? To find out –How long an algorithm takes to run –How to compare different algorithms This is done at a very abstract level This can be done before code is written Alternative: Performance analysis –Actually time each operation as the program is running Specific to the machine and the implementation of the algorithm –Can only be done after code is written

6. 6 Running Time In general, running time increases with input size Running time also affected by: –Hardware environment (processor, clock rate, memory, disk, etc.) –Software environment (operating system, programming language, compiler, interpreter, etc.) (n)

7. 7 Intuitions with Playing Cards

8. 8 Functions, Graphs of Functions Function: a rule that –Coverts inputs to outputs in a well-defined way. –This conversion process is often called mapping. –Input space called the domain –Output space called the range

9. 9 Functions, Graphs of Functions Function: a rule that –Coverts inputs to outputs in a well-defined way. –This conversion process is often called mapping. –Input space called the domain –Output space called the range Examples –Mapping of speed of bicycling to calories burned Domain: values for speed Range: values for calories –Mapping of people to names Domain: people Range: Strings

10. 10 Example: How many calories does bicycling burn? Miles/Hour vs. KiloCalories/Minute For a 150 lb rider. Green: riding on a gravel road with a mountain bike Blue: paved road riding a touring bicycle Red: racing bicyclist. From Whitt, F.R. & D. G. Wilson. 1982. Bicycling Science (second edition). http://www.frontier.iarc.uaf.edu/~cswingle/misc/exercise.phtml

11. 11 Functions and Graphs of Functions Notation  Many different kinds  f(x) is read “ f of x ”  This means a function called f takes x as an argument  f(x) = y  This means the function f takes x as input and produces y as output.  The rule for the mapping is hidden by the notation.

12. 12 Here f(x) = 7x A point on this graph can be called (x,y) or (x, f(x)) http://www.sosmath.com/algebra/logs/log1/log1.html

13. 13 A straight line is defined by the function y = ax + b a and b are constants x is variable Here y = 7x + 0 http://www.sosmath.com/algebra/logs/log1/log1.html

14. 14 Exponents and Logarithms Exponents: shorthand for multiplication Logarithms: shorthand for exponents 2 3 = 8 can be expressed as log 2 8 = 3 or we can say that the 'log with base 2 of 8' = 3. How we use these? –Difficult computational problems grow exponentially –Logarithms are useful for “ squashing ” them

15. 15 http://www.sosmath.com/algebra/logs/log1/log1.html Logarithm lets you “grab the exponent”

16. 16 The exponential function f with base a is denoted by and x is any real number. Note how much more “ quickly the graph grows ” than the linear graph of f(x) = x Example: If the base is 2 and x = 4, the function value f(4) will equal 16. A corresponding point on the graph would be (4, 16). http://www.sosmath.com/algebra/logs/log1/log1.html

17. 17 http://www.sosmath.com/algebra/logs/log1/log1.html

18. 18 Logarithmic functions are the inverse of exponential functions. If (4, 16) is a point on the graph of an exponential function, then (16, 4) would be the corresponding point on the graph of the inverse logarithmic function. http://www.sosmath.com/algebra/logs/log1/log1.html

19. 19 Illustration by Jacob Nielsen Zipf Distribution (linear and log scale)

20. 20 Other Functions Quadratic function: This is a graph of: http://www.sosmath.com/algebra/ (-2,0) (2,0)

21. 21 Summation Notation

22. 22 Summation Notation

23. 23 Intuition behind n(n+1)/2 A visual proof that 1+2+3+...+n = n(n+1)/2 Count the number of dots in T(n) but, instead of summing the numbers 1, 2, 3, … up to n we will find the total using only one multiplication and one division! http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/runsums/triNbProof.html

24. Intuition behind n(n+1)/2 A visual proof that 1+2+3+...+n = n(n+1)/2 Notice that we get a rectangle which is has the same number of rows (4) but has one extra column (5) so the rectangle is 4 by 5 it therefore contains 4x5=20 balls but we took two copies of T(4) to get this so we must have 20/2 = 10 balls in T(4), which we can easily check. http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/runsums/triNbProof.html

25. 25 Intuition behind n(n+1)/2 A visual proof that 1+2+3+...+n = n(n+1)/2 http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/runsums/triNbProof.html

26. 26 Intuition behind n(n+1)/2 T(n) = 1 + 2 + 3 +... + n = n(n + 1)/2 The same proof using algebra: http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/runsums/triNbProof.html

27. 27 Summation Notation and Code

28. 28 Iterative form vs. Closed Form

29. 29 The Seven Common Functions Constant Linear Quadratic Cubic Exponential Logarithmic n-log-n Polynomial

30. 30 Function Pecking Order In increasing order Adapted from Goodrich & Tamassia

31. 31 More Plots

32. 32 Definition of Big-Oh A running time is O(g(n)) if there exist constants n 0 > 0 and c > 0 such that for all problem sizes n > n 0, the running time for a problem of size n is at most c(g(n)). In other words, c(g(n)) is an upper bound on the running time for sufficiently large n. http://www.cs.dartmouth.edu/~farid/teaching/cs15/cs5/lectures/0519/0519.html c g(n) Example: the function f(n)=8n–2 is O(n) The running time of algorithm arrayMax is O(n)

33. 33 The Crossover Point Adapted from http://www.cs.sunysb.edu/~algorith/lectures-good/node2.html One function starts out faster for small values of n. But for n > n0, the other function is always faster.

34. 34 More formally Let f(n) and g(n) be functions mapping nonnegative integers to real numbers. f(n) is  (g(n)) if there exist positive constants n0 and c such that for all n>=n0, f(n) <= c*g(n) Other ways to say this: f(n) is order g(n) f(n) is big-Oh of g(n) f(n) is Oh of g(n) f(n)  O(g(n)) (set notation)

35. 35 Comparing Running Times Adapted from Goodrich & Tamassia

36. 36 Analysis Example: Phonebook Given: –A physical phone book Organized in alphabetical order –A name you want to look up –An algorithm in which you search through the book sequentially, from first page to last –What is the order of: The best case running time? The worst case running time? The average case running time? –What is: A better algorithm? The worst case running time for this algorithm?

37. 37 Analysis Example (Phonebook) This better algorithm is called Binary Search What is its running time? –First you look in the middle of n elements –Then you look in the middle of n/2 = ½*n elements –Then you look in the middle of ½ * ½*n elements –…–… –Continue until there is only 1 element left –Say you did this m times: ½ * ½ * ½* …*n –Then the number of repetitions is the smallest integer m such that

38. 38 Analyzing Binary Search –In the worst case, the number of repetitions is the smallest integer m such that –We can rewrite this as follows: Multiply both sides by Take the log of both sides Since m is the worst case time, the algorithm is O(logn)

39. 39 Binary Search … for Chocolate!