1. CS 312: Algorithm Analysis Lecture #7: Recurrence Relations a.k.a. Difference Equations Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.Creative Commons Attribution-Share Alike 3.0 Unported License

2. Announcements  HW #4 Due Now  Project #2  Early: next Wednesday  Due: Friday 2/1  Much more challenging than Project #1 – plan accordingly  Career Fair!

3. Objectives  Goal: Analyze Divide and Conquer recursive algorithms using recurrence relations (RRs)  Also known as Difference Equations  Working up to such RRs  Today: Focus on a special type of RRs  Homogeneous  Linear  Constant Coefficients  Leading up to a proof of the Master Theorem

4. Recall: Analysis Framework Given a problem,  Identify platform’s elementary operations (sometimes implicit)  Formulate solution as an algorithm  Define the measure of the input size  Measure time efficiency by counting the number of times an elementary operation is executed  Measure space efficiency by counting the number of memory units consumed by the algorithm  The efficiency of some algorithms may differ significantly for inputs of the same size.  Distinguish among worst-case, average-case, and best-case efficiencies. Choose one.  Plot efficiency vs. input size  Establish order of growth. Use asymptotic notation. n0n0 c 1 g(n) c 2 g(n) f(n) Where is the difficulty for Analyzing Recursive Functions?

5. Nonrecursive Algorithms Example: Element Uniqueness Problem, check whether all elements in an array are distinct. function UniqueElements(A[0…n-1]) for i=0 to n-2 for j = i+1 to n-1 if A[i]=A[j] return false return true Define input size Define elementary operation Distinguish worst-case

6. Recurrence Relations

7. Recursive Algorithms Example: Compute Factorial function N! function Factorial(N) if N=0 return 1 else return Factorial(N-1)*N Assume 32-bit integers Define input size: _____ Define elementary operations: Focus on worst-case What sequence is generated by this recurrence relation? In order to answer that question we need an initial condition! Now we can build a table of values: How long to compute C(1,000,000)? Want: C(N) in closed form for quick computation. N C(N)

8. Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one. How to solve?

9. Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

10. Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

11. Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

12. Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

13. Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one. Can you figure out an explicit (closed form) formula for this sequence? As before, build a table, recognize the pattern OR Use substitution, recognize the pattern OR Appeal to theory of recurrence relations n C(n)

14. Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one. Can you figure out an explicit (closed form) formula for this sequence? As before, build a table, recognize the pattern OR Use substitution, recognize the pattern OR Appeal to theory of recurrence relations n C(n)

15. Substitution 123456789 k y(k)

16. Recurrence Relations The most general form for our purposes:

17. Constant Coefficients

18. Example Capital growth in your savings account y(n)= $$ in account end of day n i = interest rate earned each day (at start of day) D(n) = Deposit on day n W(n) = Withdraw on day n You try it! Write a recurrence relation for y(n)

19. Order of Recurrence Relations What is the order of this RR?

20. Forcing Function

21. Homogeneous

22. What kind of RR is this? 123456789 k y(k)

23. Terminology Review Note the alternate notations.

24. Terminology Review Note the alternate notations.

25. Existence and Uniqueness Theorem

26. Starting Point

27. Example Goal: find solution t n

28. Example Goal: find solution t n

29. General Solution

30. Solution?

33. Assignment  Read: Recurrence Relations Notes, Part II  HW #5: Part I (Section 1.2) Exercises in the RR notes.

34. End

35. Extra: Big Picture  Look at the big picture: functions as “systems”

36. Recurrence Relations So why should we care about recurrence relations? Generalizes the idea of a function to an operator or dynamic system A function is like a table that takes a number in and gives a number out. f xy xy 1 2 3 4 2 4 8 16 123456789 x y 123456789 x y

37. Recurrence Relations Functions can also take in a vector of numbers or yield a vector out. f x1x2y1y2y3 11113 13225 12136 14243 y1y1 x1x1 x2x2 y2y2 x1x1 x2x2 y3y3 x1x1 x2x2

38. Recurrence Relations What if the input and output vectors of a function were infinitely long? f Then x and y themselves are functions! f becomes a map from one function to another! We call maps that take functions as inputs or generate functions as outputs operators, or systems. An operator is like a rule, it’s the mathematical description of a subroutine! 1234567 8 9 k x 1234567 8 9 k y

39. Recurrence Relations S x(k) y(k) 1234567 8 9 k x 1234567 8 9 k y

40. Recurrence Relations S x(k) y(k) Unlike a simple function, an operator can have local variables that store intermediate calculations—just like a subroutine. In other words, S is a map that can have memory!

41. Recurrence Relations S x(k) y(k) Subroutine Description static memory: w = 0 function S (x,k) if k=0 then y  y o else y  w + x w  y return y Math Description y(0)=y o

42. Math Description Subroutine Description static memory: w = 0 function S (x,k) if k=0 then y  y o else y  5w + x w  y return y Recurrence Relations S x(k) y(k) y(0)=y o

43. Subroutine Description static memory: w 0, w 1 = 0 function S (x,k) if k=0 then y  y o else if k=1 then y  y 1 else y  a 1 w 1 + a o w 0 +x w 0  w 1 w 1  y return y Math Description Recurrence Relations S What if more than one number is stored in memory? x(k) y(k) y(0)=y o, y(1) = y 1