Mathematical Analysis of Recursive Algorithm CSG3F3 Lecture 7

2 Set up a recurrence relations and initial condition

3 Example 1 ALGORITHM F(n) //computes n! recursively //input: a nonnegative integer n //output: the value of n! if n=0 return 1 else return F(n-1)*n

4 Recurrence relation and initial condition for the algorithm’s number of multiplication M(n): M(n)=M(n-1)+1for n>0, M(0)=0

5 M(n)=M(n-1)+1substitute M(n-1)=M(n-2)+1 =[M(n-2)+1]+1=M(n-2)+2 substitute M(n-2)=M(n-3)+1 =[M(n-3)+1]+2=M(n-3)+3 General formula: M(n)=M(n-i)+i

6 For n=0, M(n)=M(n-1)+1=…=M(n-i)+i=… =M(n-n)+n=n

7 Techniques for solving recurrence Substitution Characteristic equation Induction

8 Solving Recurrences by Substitution Recurrence: Work backward:

9 Substitution (cont.) Substitution:

Problem 1 10

11 Problem 2 ALGORITHM Power(x, n) // Computes x to the power of n. // Input: x is a real number, n an integer >= 0 // Output: Returns x to the power of n. if n=0 return 1 else return x * Power(x, n-­1)

12 Problem 2 (cont.) Analysis Basic operation: multiplication M(n) = 1 + M(n-­1): recurrence relation M(0) = 0 : initial condition

13 Problem 2 (cont.) Solving the recurrence relation M(n) = 1 + M(n-­1) = 1 + [M(n-­2) + 1 ] = 2 + M(n -2) = 2 + [M(n -3) + 1 ] = 3 + M(n -3) = : = k + M(n-­k)

14 Problem 2 (cont.) M(n) = k + M(n-­k) = n + M(n-n) = n + M(0) = n + 0 = n Order of growth M(n)  O(n)

15 General Plan 1.Decide on parameter n indicating input size 2.Identify algorithm’s basic operation 3.Determine worst, best, and average case for input of size n 4.Set up a recurrence relation and initial condition for C(n)-the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls). 5.Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution

16 Problem 2: Binary Recursive Problem: investigate a recursive version of binary algorithm, which finds the number of binary digits in the binary representation of a positive decimal integer n. Recursive relation for the number of addition made by the algorithm. A(n)=A[n/2] +1for n>1 A(1)=0initial condition for n =1, no addition is made because no recursive call is executed.

17 Problem 3: Binary Recursive Algorithm BinRec(n) //input: a positive decimal integer n //output: the number of binary digits in n’s binary representation If n=1 return 1 Else return BinRec([n/2])+1

18 A(2 k ) = A(2 k-1 ) + 1 = [A(2 k-2 ) + 1] + 1 = A(2 k-2 ) + 2 = [A(2 k-3 ) + 1] + 2 = A(2 k-3 ) + 3 …… = A(2 k-i ) + i …… = A(2 k-k ) + k = k A(2 0 ) = A(1) = 0 = log 2 n n = 2 k A(n) = log 2 n  O(log n)

19 Exercises Set up and solve a recurrence relation for the number of times to the following algorithms

20 Tower of Hanoi ALGORITHM Move(n, first, temp, last) // Input: a positive integer n // first, temp, and last are tower numbers // Output: a list of moves for n disks from fir st to // last using temp as auxiliary. If n = 1 output "Move 1 disk from" first "to" last else Move(n-­1, first, last, temp) output "Move 1 disk from" first "to" last Move(n-­1, temp, first, last)

21 T(n)=2T(n/2)+n-1n >1, n a power of 2 T(1)=0 ALGORITHM S(n) //input: a positive integer n //output: the sum of the first n cubes if n=1 return 1 else return S(n-1)+n*n*n

22 Characteristic Equation Homogeneous Linear Recurrences k and a i are constant It’s called “linear” because every term t i appears only to the first power It’s called “homogeneous” because the linear combination is equal to 0.

23 Characteristic (cont.) The following are homogeneous linear recurrence equations

24 Characteristic (cont.) The Fibonacci Sequence is defined as follows: Subtract t n-1 and t n-2 from both sides, we get homogeneous linear recurrence equation

25 Characteristic (cont.) Definition

26 Characteristic (cont.) Example

27 Characteristic (cont.) Theorem. Let the homogeneous linear recurrence equation with constant coefficients be given. If its characteristic equation has k distinct solutions r 1, r 2,…,r k, then the only solutions to the recurrence are

28 Characteristic (cont.) Consider the following recurrence 1.Obtain the characteristic equation

29 Characteristic (cont.) 2.Solve the characteristic equation 3.Apply the theorem to get the general solution: 4.Determine the values of the constants

30 Characteristic (cont.) The solution to this system is c 1 =1/5, c 2 =-1/5 5.Substitute the constant into the general solution to obtain the particular solution:

Exercise Fibonacci Sequence 31

32 Multiplicity root Theorem. Let r be a root of multiplicity m of the characteristic equation for a homogeneous linear recurrence with constant coefficients. Then are all solutions to the recurrence.

Exercises 33

34 Characteristic (cont.) Nonhomogeneous Linear Recurrences k and a i are constant, f(n) is a function other than the zero function. Develop a method for solving the common case where b is a constant and p(n) is a polynomial in n.

35 Characteristic (cont.) Theorem. A nonhomogeneous linear recurrence of the form can be transformed into homogeneous linear recurrence where d is the degree of p(n).

Characteristic (cont.)

Characteristic (cont.) 37 Cont’d a. Replace n with n-1 in the original recurrence so that the recurrence is expressed with 4 n-1 on the right: b. Divide the original recurrence by 4 so that the recurrence is expressed in another way with 4 n-1 on the right: c. To get rid of the term 4 n-1, subtracting the recurrence a) from b): Multiply by 4 to get rid of the fractions

38 Characteristic (cont.) Solve the following recurrence 1.Obtain the characteristic eq for the corresponding homogeneous eq:

39 Characteristic (cont.) 2.Obtain a term from the nonhomogeneous part of the recurrence the term is

40 Characteristic (cont.) 3.Apply theorem to obtain the characteristic eq. 4.Solve the characteristic equation the roots are r=3 and r=4, and the root r=4 has multiplicity 2

41 Characteristic (cont.) 5.Apply theorem to get the general solution We must find another initial condition. Because and t 1 =12,

42 Characteristic (cont.) 6.Determine the value of the constants 7.Substitute the constants into the general solution to obtain

Exercises 43

44 Change of variables The recurrence relation:

45 Change of variables 1.Set n=2 k, which means k= 2 log n 2.Substitute 2 k for n to obtain

46 Change of variables 3.Next set: t k = T(2 k ) in 2) to obtain the new recurrence t k = t k General solution t k = c 1 + c 2 k

47 Change of variables 5.Substitute T(2 k ) for t k in the general solution: T(2 k ) = c 1 + c 2 k 6.Substitute n for 2 k and lg n for k : T(n) = c 1 + c 2 lg n

48 Change of variables T(n) = 1 + lg n

Exercises 49

50 Neapolitan, Richard. Kumarss Naimipour. Foundations of algorithms. D.C Heath and Company. 1996

51 Next.. Brute Force Technique