1 Recurrences Algorithms Jay Urbain, PhD Credits: Discrete Mathematics and Its Applications, by Kenneth Rosen The Design and Analysis of Algorithms, by Levitan

2 Overview We are interested in determining the running time behavior of algorithms expressed as bounds using Θ, O, or Ω representation.

3 Recursive Algorithms Recurrence relations result naturally from the analysis of recursive algorithms. Solving recurrence relations yields a closed-end formula for calculation of run time. Example: the recursive n! algorithm: int factorial (int n) { if (n == 1) return 1; return factorial (n-1) * n; }

4 n! int factorial (int n) { if (n == 1) return 1; return factorial (n-1) * n; } The running time, T(n), can be defined as: T(1) = 1*T(n) = T(n-1) + 1 for all n>1 where 1 is the running time for each execution of the factorial function.

5 n! Solving the recurrence by recognizing the summation leads to closed- end formula: T(n) = T(1) = = n The run time for execution of factorial can now be directly computed for a given n. T(10) = 10. More importantly, when expressing the worst case run time bounds we can then write: T(n) = O(1) if n=1; T(n) = O(n) if n>1

6 Sequences and Recurrence Relations Sequence - a numerical, ordered list of numbers T - a recurrence function defining a sequence n - a parameter to function T T(n) - the sequence term generated by function T for parameter n

7 Sequences and Recurrence Relations Is T(1) = 1 necessary?

8 Sequences and Recurrence Relations Inductive proofs on the recurrence: T(n) = T(n-1) + 1 Typically assume the case for parameter n that generates a sequence term for recurrence T(n) and prove: T(n+1)=T((n+1)-1)+1. For above recurrence: T(n) = T(floor(n/4)) + n What parameter value generates the next sequence term following T(n)?

9 Defining a sequence Either T n or T(n) which stresses that a sequence is a function where T(n) is the generic nth term. Defining a sequence (two ways) –Closed-end formula of generic term as a function of n, T(n) = 2n for n>=0 –Recurrence equation relating generic term to one or more other sequence terms combined with one or more explicit values of first term(s). T(n) = T(n-1) + n for n>1 // generic term defined by other sequence terms, i.e., recurrence T(0) = 0//initial condition

10 Example: Fibonacci Fibonacci sequence definition: int fibonacci (int n) { if( n==0 || n==1 ) return n; return fibonacci(n-1) + fibonacci(n-1); } has the corresponding recurrence: F(0)=0 Initial F(1)=1 Initial F(n) = F(n-1)+F(n-2) for n>1 Recurrence

11 Solve Recurrence Relation Find closed-end formula for generic nth term of sequence that satisfies both the recurrence term and initial condition or prove that the sequence does not exist. Example T(n) = T(n-1) + n for n>1B1 T(0) = 0B2 Solution of B1 subject to initial condition B2 is: T(n) = n(n+1) for n>=0 B3 2

12 Verify closed-end solution Substitute solution into recurrence formula to check that equality holds or use induction to prove. B1 must hold for every n>0:

13 Solution to Recurrence Particular solution to recurrence - a specific sequence that satisfies recurrence. –B3 is a particular solution for B1 and B2. General solution to recurrence - formula that specifies all sequences, typically includes arbitrary constants. A general solution for recurrence B1 is: T(n) = c + n(n+1) for n>=0 2 choosing different values for c gives all solutions to B1. No universal method but common techniques for a variety of recurrences!

14 Forward substitution Forward substitution for finding exact closed-end equation for T(n) - limited to simple recurrences. Start with initial term(s) given by initial conditions, generate first few terms in hope of seeing a pattern that can be expressed by a closed- end formula. Example T(n) = 2T(n-1)+1 for n>1 B5 T(1)=1 B6 Generate T(1) = 1 = T(2) = 2T(2-1)+1 = 2T(1)+1 = 2*1+1 = 3 = T(3) = 2T(3-1)+1 = 2T(2)+1 = 2*3+1 = 7 = T(4) = 2T(4-1)+1 = 2T(3)+1 = 2*7+1 = 15 = Hope Observe pattern that suggests T(n) = 2 n -1 for n=1,2,3,4

15 Forward substitution Prove T(n) = 2 n -1 correct by substitution or induction

Backward Substituion express T(n-1) as a function of T(n-2) substitute result to give T(n) as a function of T(n-2) repeating for T(n-2) gives T(n) as function of T(n-3) hope to see a pattern to express T(n) as function of T(n-i) for i=1,2,... select i to make n-i reach the initial condition closed-end formula –use standard summation formula from Appendix A will often lead to closed-end formula –when lucky, get form similar to: T(n) = Base case + f(n), and can solve by substitution 16

Backward Substituion 17

Backward Substitution 18

19 Common Recurrence Types in Algorithmic Analysis Decrease-by-one algorithms exploits relationship between instance of size n and smaller instance of size n-1. Decrease-by-a-constant factor algorithms reduce instance of size n to an instance of size n/b (b=2 for many such algorithms), solving the smaller instance recursively. If necessary, the solution of the smaller instance is combined to a solution of the given instance. Divide-and-conquer divides a given instance into smaller instances, solving each recursively. If necessary solutions to smaller instances are combined to give a solution to a given instance.

20 Decrease-by-one algorithms Decrease-by-one algorithms exploits relationship between instance of size n and smaller instance of size n-1.

21 Decrease-by-one algorithms Typical form of recurrence equation, where f(n) is the time to reduce an instance to a smaller one is:

22 Specific functions - f(n)

23 Decrease-by-a-constant factor algorithms Reduce instance of size n to an instance of size n/b (b=2 for many such algorithms), solving the smaller instance recursively. If necessary, the solution of the smaller instance is extended to a solution of the given instance.

24 Decrease-by-a-constant factor algorithms – Binary Search

25 Divide and Conquer Divides a given instance into smaller instances, solving each recursively. If necessary solutions to smaller instances are combined to give a solution to a given instance.

26

Merge Sort 27

Merge Sort 28

Merge Sort 29

General form of recurrence equation n is the problem size B>=2 the number of problem divisions n/b is the size of the smaller problem after division A>=1 is the number of sub-problem solutions f(n) is the time to divide an instance to a smaller ones and combine the solutions T(n) = aT(n/b) + f(n) 30