1 Recursive Algorithm Analysis Dr. Ying Lu RAIK 283: Data Structures & Algorithms September 13, 2012

2 b Giving credit where credit is due: Most of the lecture notes are based on the slides from the Textbook’s companion websiteMost of the lecture notes are based on the slides from the Textbook’s companion websitehttp:// Several slides are from Hsu Wen Jing of the National University of SingaporeSeveral slides are from Hsu Wen Jing of the National University of Singapore I have modified them and added new slidesI have modified them and added new slides RAIK 283: Data Structures & Algorithms

3 Example: a recursive algorithm b Algorithm: if n=0 then F(n) := 1 else F(n) := F(n-1) * n return F(n)

4 Algorithm F(n) // Compute the n th Fibonacci number recursively //Input: A nonnegative integer n //Output: the n th Fibonacci number if n  1 return n else return F(n-1) + F(n-2) Example: another recursive algorithm

5 Recurrence Relation b Recurrence Relation

6 Recurrence Relation b Recurrence Relation: an equation or inequality that describes a function in terms of its value on smaller inputs

7 Example: a recursive algorithm b Algorithm: if n=0 then F(n) := 1 else F(n) := F(n-1) * n return F(n) b What does this algorithm compute?

8 Example: a recursive algorithm b Algorithm: if n=0 then F(n) := 1 else F(n) := F(n-1) * n return F(n) b What does this algorithm compute? b What’s the basic operation of this algorithm?

9 Example: recursive evaluation of n ! b Recursive definition of n!: b Algorithm: if n=0 then F(n) := 1 else F(n) := F(n-1) * n return F(n) b M(n): number of multiplications to compute n! with this recursive algorithm

10 Example: recursive evaluation of n ! b Recursive definition of n!: b Algorithm: if n=0 then F(n) := 1 else F(n) := F(n-1) * n return F(n) b M(n): number of multiplications to compute n! with this recursive algorithm b Could we establish a recurrence relation for deriving M(n)?

11 Example: recursive evaluation of n ! b Definition: n ! = 1*2*…*(n-1)*n b Recursive definition of n!: b Algorithm: if n=0 then F(n) := 1 else F(n) := F(n-1) * n return F(n) b M(n) = M(n-1) + 1 b Initial Condition: M(0) = ?

12 Example: recursive evaluation of n ! b Recursive definition of n!: b Algorithm: if n=0 then F(n) := 1 else F(n) := F(n-1) * n return F(n) b M(n) = M(n-1) + 1 b Initial condition: M(0) = 0 b Explicit formula for M(n) in terms of n only?

13 Time efficiency of recursive algorithms Steps in analysis of recursive algorithms: b Decide on parameter n indicating input size b Identify algorithm’s basic operation b Determine worst, average, and best case for inputs of size n b Set up a recurrence relation and initial condition(s) for C(n)-the number of times the basic operation will be executed for an input of size n b Solve the recurrence to obtain a closed form or determine the order of growth of the solution (see Appendix B)

14 EXAMPLE: tower of hanoi b Problem: Given three pegs (A, B, C) and n disks of different sizesGiven three pegs (A, B, C) and n disks of different sizes Initially, all the disks are on peg A in order of size, the largest on the bottom and the smallest on topInitially, all the disks are on peg A in order of size, the largest on the bottom and the smallest on top The goal is to move all the disks to peg C using peg B as an auxiliaryThe goal is to move all the disks to peg C using peg B as an auxiliary Only 1 disk can be moved at a time, and a larger disk cannot be placed on top of a smaller oneOnly 1 disk can be moved at a time, and a larger disk cannot be placed on top of a smaller one AC B n disks

15 EXAMPLE: tower of hanoi b Design a recursive algorithm to solve this problem: Given three pegs (A, B, C) and n disks of different sizesGiven three pegs (A, B, C) and n disks of different sizes Initially, all the disks are on peg A in order of size, the largest on the bottom and the smallest on topInitially, all the disks are on peg A in order of size, the largest on the bottom and the smallest on top The goal is to move all the disks to peg C using peg B as an auxiliaryThe goal is to move all the disks to peg C using peg B as an auxiliary Only 1 disk can be moved at a time, and a larger disk cannot be placed on top of a smaller oneOnly 1 disk can be moved at a time, and a larger disk cannot be placed on top of a smaller one AC B n disks

16 EXAMPLE: tower of hanoi b Step 1: Solve simple case when n<=1? Just trivial AC B AC B Move(A, C)

17 EXAMPLE: tower of hanoi b Step 2: Assume that a smaller instance can be solved, i.e. can move n-1 disks. Then? AC B AC B AC B

18 EXAMPLE: tower of hanoi AC B AC B AC B AC B

19 EXAMPLE: tower of hanoi AC B AC B AC B AC B TOWER(n, A, B, C)

20 EXAMPLE: tower of hanoi AC B AC B AC B AC B TOWER(n, A, B, C) TOWER(n-1, A, C, B) Move(A, C) TOWER(n-1, B, A, C)

21 EXAMPLE: tower of hanoi TOWER(n, A, B, C) { TOWER(n-1, A, C, B); Move(A, C); TOWER(n-1, B, A, C) } if n<1 return;

22 EXAMPLE: tower of hanoi TOWER(n, A, B, C) { TOWER(n-1, A, C, B); Move(A, C); TOWER(n-1, B, A, C) } if n<1 return; b Algorithm analysis: b Input size? Basic operation?

23 EXAMPLE: tower of hanoi TOWER(n, A, B, C) { TOWER(n-1, A, C, B); Move(A, C); TOWER(n-1, B, A, C) } if n<1 return; b Algorithm analysis: b Do we need to differentiate best case, worst case & average case for inputs of size n?

24 EXAMPLE: tower of hanoi TOWER(n, A, B, C) { TOWER(n-1, A, C, B); Move(A, C); TOWER(n-1, B, A, C) } if n<1 return; b Algorithm analysis: b Set up a recurrence relation and initial condition(s) for C(n)

25 EXAMPLE: tower of hanoi TOWER(n, A, B, C) { TOWER(n-1, A, C, B); Move(A, C); TOWER(n-1, B, A, C) } if n<1 return; b Algorithm analysis: b C(n) = 2C(n-1)+1

26 In-Class Exercise b P. 76 Problem (c): solve this recurrence relation: x(n) = x(n-1) + n for n>0, x(0)=0 b P. 77 Problem 2.4.4: consider the following recursive algorithm: Algorithm Q(n)Algorithm Q(n) // Input: A positive integer n If n = 1 return 1 else return Q(n-1) + 2 * n – 1 A. Set up a recurrence relation for this function’s values and solve it to determine what this algorithm computesA. Set up a recurrence relation for this function’s values and solve it to determine what this algorithm computes B. Set up a recurrence relation for the number of multiplications made by this algorithm and solve it.B. Set up a recurrence relation for the number of multiplications made by this algorithm and solve it. C. Set up a recurrence relation for the number of additions/subtractions made by this algorithm and solve it.C. Set up a recurrence relation for the number of additions/subtractions made by this algorithm and solve it.

27 Example: BinRec(n) Algorithm BinRec(n) //Input: A positive decimal integer n //Output: The number of binary digits in n’s binary representation if n = 1 return 1 else return BinRec(  n/2  ) + 1

28 Smoothness rule If T(n)   (f(n)) for values of n that are powers of b, where b  2, then T(n)   (f(n))

29 Example: BinRec(n) Algorithm BinRec(n) //Input: A positive decimal integer n //Output: The number of binary digits in n’s binary representation if n = 1 return 1 else return BinRec(  n/2  ) + 1 If C(n)   (f(n)) for values of n that are powers of b, where b  2, then C(n)   (f(n))

30 Fibonacci numbers b The Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … b Fibonacci recurrence: F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1 2nd order linear homogeneous recurrence relation with constant coefficients

31 Solving linear homogeneous recurrence relations with constant coefficients b Easy first: 1 st order LHRRCCs: C(n) = a C(n -1) C(0) = t … Solution: C(n) = t a n b Extrapolate to 2 nd order L(n) = a L(n-1) + b L(n-2) … A solution?: L(n) = ? b Characteristic equation (quadratic) b Solve to obtain roots r 1 and r 2 (quadratic formula) quadratic formulaquadratic formula b General solution to RR: linear combination of r 1 n and r 2 n b Particular solution: use initial conditions

32 Solving linear homogeneous recurrence relations with constant coefficients b Easy first: 1 st order LHRRCCs: C(n) = a C(n -1) C(0) = t … Solution: C(n) = t a n b Extrapolate to 2 nd order L(n) = a L(n-1) + b L(n-2) … A solution?: L(n) = ? b Characteristic equation (quadratic) b Solve to obtain roots r 1 and r 2 (quadratic formula) quadratic formulaquadratic formula b General solution to RR: linear combination of r 1 n and r 2 n b Particular solution: use initial conditions b Explicit Formula for Fibonacci Number: F(n) = F(n-1) +F(n-2)

33 1. Definition based recursive algorithm Computing Fibonacci numbers Algorithm F(n) // Compute the n th Fibonacci number recursively //Input: A nonnegative integer n //Output: the n th Fibonacci number if n  1 return n else return F(n-1) + F(n-2)

34 2. Nonrecursive brute-force algorithm Computing Fibonacci numbers Algorithm Fib(n) // Compute the n th Fibonacci number iteratively //Input: A nonnegative integer n //Output: the n th Fibonacci number F[0]  0; F[1]  1 for i  2 to n do F[i]  F[i-1] + F[i-2] return F[n]

35 Computing Fibonacci numbers 3. Explicit formula algorithm Special care in its implementation: Intermediate results are irrational numbers Their approximations in the computer are accurate enough Final round-off yields a correct result

36 In-Class Exercises b What is the explicit formula for A(n)? A(n) = 3A(n-1) – 2A(n-2) A(0) = 1 A(1) = 3 b P Climbing stairs: Find the number of different ways to climb an n-stair stair-case if each step is either one or two stairs. (For example, a 3- stair staircase can be climbed three ways: 1-1-1, 1- 2, and 2-1.)