1. 6.Advanced Counting Techniques 1 Copyright M.R.K. Krishna Rao 2003 Ch 6. Recurrence Relations A recurrence relation for the sequence {a n } is an equation that expresses a n in terms of one or more of the previous terms of the sequence, namely, a 0, a 1, …, a n-1, for all integers n with n  n 0, where n 0 is a nonnegative integer. A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.

2. 6.Advanced Counting Techniques 2 Copyright M.R.K. Krishna Rao 2003 Recurrence Relations In other words, a recurrence relation is like a recursively defined sequence, but without specifying any initial values (initial conditions). Therefore, the same recurrence relation can have (and usually has) multiple solutions. If both the initial conditions and the recurrence relation are specified, then the sequence is uniquely determined.

3. 6.Advanced Counting Techniques 3 Copyright M.R.K. Krishna Rao 2003 Recurrence Relations Example: Consider the recurrence relation a n = 2a n-1 – a n-2 for n = 2, 3, 4, … Is the sequence {a n } with a n =3n a solution of this recurrence relation? For n  2 we see that 2a n-1 – a n-2 = 2(3(n – 1)) – 3(n – 2) = 3n = a n. Therefore, {a n } with a n =3n is a solution of the recurrence relation.

4. 6.Advanced Counting Techniques 4 Copyright M.R.K. Krishna Rao 2003 Recurrence Relations Is the sequence {a n } with a n =5 a solution of the same recurrence relation? For n  2 we see that 2a n-1 – a n-2 = 2  5 - 5 = 5 = a n. Therefore, {a n } with a n =5 is also a solution of the recurrence relation. In other words, the same recurrence relation can have (and usually has) multiple solutions. If both the initial conditions and the recurrence relation are specified, then the sequence is uniquely determined.

5. 6.Advanced Counting Techniques 5 Copyright M.R.K. Krishna Rao 2003 Modeling with Recurrence Relations Example: Someone deposits SR 10,000 in a savings account at a bank yielding 5% per year with interest compounded annually. How much money will be in the account after 30 years? Solution: Let P n denote the amount in the account after n years. How can we determine P n on the basis of P n-1 ?

6. 6.Advanced Counting Techniques 6 Copyright M.R.K. Krishna Rao 2003 Modeling with Recurrence Relations We can derive the following recurrence relation: P n = P n-1 + 0.05P n-1 = 1.05P n-1. The initial condition is P 0 = 10,000. Then we have: P 1 = 1.05P 0 P 2 = 1.05P 1 = (1.05) 2 P 0 P 3 = 1.05P 2 = (1.05) 3 P 0 … P n = 1.05P n-1 = (1.05) n P 0 We now have a formula to calculate P n for any natural number n and can avoid the iteration.

7. 6.Advanced Counting Techniques 7 Copyright M.R.K. Krishna Rao 2003 Modeling with Recurrence Relations Let us use this formula to find P 30 under the initial condition P 0 = 10,000: P 30 = (1.05) 30  10,000 = 43,219.42 After 30 years, the account contains SR 43,219.42.

8. 6.Advanced Counting Techniques 8 Copyright M.R.K. Krishna Rao 2003 Rabbit Reproduction MonthAdult pairsYoungTotal 1011 2011 3112 4123 5235 6358 Fibonacci defined a sequence when observing the reproduction of rabbits. Starting with one male/female pair of newborn rabbits, he came up with the following: Question: After n months, how many pairs of mature rabbits will there be?

9. 6.Advanced Counting Techniques 9 Copyright M.R.K. Krishna Rao 2003 Rabbit Reproduction - Fibonacci numbers This sequence of numbers {f n } satisfies the recurrence relation f n = f n-1 + f n-2, n  3 With initial conditions f 1 = 1 and f 2 = 1. The sequence of Fibonacci numbers are: 1, 1, 2, 3, 5, 8, 13, 21, ……

10. 6.Advanced Counting Techniques 10 Copyright M.R.K. Krishna Rao 2003 Modeling with Recurrence Relations The Legend: In an ancient city, so the legend goes, monks in a temple have to move a pile of 64 sacred disks from one location to another. The disks are fragile; only one can be carried at a time. A disk may not be placed on top of a smaller, less valuable disk. And, there is only one other location in the temple (besides the original and destination locations) sacred enough that a pile of disks can be placed there. So, the monks start moving disks back and forth, between the original pile, the pile at the new location, and the intermediate location, always keeping the piles in order (largest on the bottom, smallest on the top). The legend is that, before the monks make the final move to complete the new pile in the new location, the temple will turn to dust and the world will end. Should we worry about truth of this legend?

11. 6.Advanced Counting Techniques 11 Copyright M.R.K. Krishna Rao 2003 Towers of Hanoi Question: What is the minimum # of moves required to shift N disks from one spoke to another? Solution: (1) Transfer n-1 disks to peg 3 in H n-1 steps. (2) Transfer the largest disk to peg 2 in 1 step. (3) Now, transfer the n-1 disks from peg 3 to peg 2. H n = H n-1 + 1 + H n-1 = 2H n-1 + 1 No need to worry! -- it will take more than 500 billion years!

12. 6.Advanced Counting Techniques 12 Copyright M.R.K. Krishna Rao 2003 Example Find a recurrence relation and give initial conditions for the number of bit strings of length n that do not have two consecutive 0s. How many such bit strings are there of length eight?

13. 6.Advanced Counting Techniques 13 Copyright M.R.K. Krishna Rao 2003 Solution Let a n be the number of bit strings of length n that do not have two consecutive 0s. For n  3, a n = a n-1 + a n-2 with initial conditions a 1 = 2 and a 2 = 3. Hence, there are 55 bit strings of length 8 with no two consecutive 0s. (Seq. 2, 3, 5, 8, 13, 21, 34, 55…) a n-1 a n-2 First bit Second bit  0 0 1 1

14. 6.Advanced Counting Techniques 14 Copyright M.R.K. Krishna Rao 2003 Example: (Codeword Enumeration) A string of decimal digits is considered as a valid codeword if it contains an even number of 0 digits. For instance, 12340407869 is valid, whereas 120987045608 is not valid. Let a n be the number of valid n-digit codewords. Find a recurrence relation for a n.

15. 6.Advanced Counting Techniques 15 Copyright M.R.K. Krishna Rao 2003 Codeword Enumeration (cont.) There are 2 cases: Case (a): First bit is non-zero (9 ways): –It’s okay if the remaining n-1 bits contain an even number of 0: a n-1 such strings. Case (b): First bit is zero: –It’s okay if the remaining n-1 bits contain an ODD number of 0: (10 n-1  a n-1 ) such strings. a n = 9  a n-1 + (10 n-1  a n-1 ) = 8  a n-1 + 10 n-1 with initial condition a 1 = 9.

16. 6.Advanced Counting Techniques 16 Copyright M.R.K. Krishna Rao 2003 Sterling Numbers Example: (Sterling Numbers of second kind) Let S n,r = the number of ways a set of size n can be partitioned into r subsets. Let’s find S 4,1, S 4,2, S 4,3 and S 4,4. S 4,1 = 1 -- only 1 way S 4,2 = 7 -- A+BCD, B+ACD, C+ABD, D+ABC, AB+CD, AC+BD, AD+BC S 4,3 = 6-- AB+C+D, AC+B+D, AD+B+C,… S 4,4 = 1-- only 1 way

17. 6.Advanced Counting Techniques 17 Copyright M.R.K. Krishna Rao 2003 Sterling Numbers… Let’s find a recurrence relation for S n,r : Partition of {x 1, x 2, …, x n } into r subsets = Partition of {x 1, x 2, …, x n } into r subsets where one of the subsets is {x n } + Partition of {x 1, x 2, …, x n } into r subsets where none of the subsets is {x n } Therefore, S n,r, = S n-1,r-1, + rS n-1,r Blackboard Examples: 8 – Catalan numbers. Blackboard Exercises: 2,9,13,18,21,29,37

18. 6.Advanced Counting Techniques 18 Copyright M.R.K. Krishna Rao 2003 6.2 Solving Recurrence Relations In general, we would prefer to have an explicit formula to compute the value of a n rather than conducting n iterations. Two main methods: 1.Iteration (or telescoping). 2.Method for linear homogeneous recurrence relations with constant coefficients.

19. 6.Advanced Counting Techniques 19 Copyright M.R.K. Krishna Rao 2003 Method 1: Iteration Problem: Given a recursive expression with initial conditions a 0, a 1, Try to express a n without dependence on previous terms. Example : a n = 2a n-1 for n > 1, with initial condition a 0 = 1 Solution: a n = 2 n

20. 6.Advanced Counting Techniques 20 Copyright M.R.K. Krishna Rao 2003 More on the iteration method Example: Deer Population growth Deer population d n at time n Initial condition: d 0 = 1000 Increase from time n-1 to time n is 10%. Therefore the recursive function is d n – d n-1 = 0.1d n-1  d n = 1.1d n-1 Solution: d n = 1000(1.1) n

21. 6.Advanced Counting Techniques 21 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: a n =2a n-1

22. 6.Advanced Counting Techniques 22 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: a n =2a n-1 =2 2 a n-2

23. 6.Advanced Counting Techniques 23 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: a n =2a n-1 =2 2 a n-2 =2 3 a n-3

24. 6.Advanced Counting Techniques 24 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: a n =2a n-1 =2 2 a n-2 =2 3 a n-3 =…

25. 6.Advanced Counting Techniques 25 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: a n =2a n-1 =2 2 a n-2 =2 3 a n-3 =… =2 i a n-i

26. 6.Advanced Counting Techniques 26 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: a n =2a n-1 =2 2 a n-2 =2 3 a n-3 =… =2 i a n-i =…

27. 6.Advanced Counting Techniques 27 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: a n =2a n-1 =2 2 a n-2 =2 3 a n-3 =… =2 i a n-i =… =2 n a 0

28. 6.Advanced Counting Techniques 28 Copyright M.R.K. Krishna Rao 2003 Telescope Example Find a closed solution to a n = 2a n-1, a 0 = 3: Plug in a 0 = 3 for final answer: a n = 3 · 2 n a n =2a n-1 =2 2 a n-2 =2 3 a n-3 =… =2 i a n-i =… =2 n a 0

29. 6.Advanced Counting Techniques 29 Copyright M.R.K. Krishna Rao 2003 Method 2: Linear Recurrences The only case for which telescoping works with a high probability is when the recurrence gives the next value in terms of a single previous value. But… There is a class of recurrence relations which can be solved analytically in general. These are called linear recurrences and include the Fibonacci recurrence.

30. 6.Advanced Counting Techniques 30 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Definition: A linear homogeneous recurrence relation of degree k with constant coefficients is a recurrence relation of the form: a n = c 1 a n-1 + c 2 a n-2 + … + c k a n-k, Where c 1, c 2, …, c k are real numbers, and c k  0. A sequence satisfying such a recurrence relation is uniquely determined by the recurrence relation and the k initial conditions a 0 = C 0, a 1 = C 1, a 2 = C 2, …, a k-1 = C k-1.

31. 6.Advanced Counting Techniques 31 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Examples: The recurrence relation P n = (1.05)P n-1 is a linear homogeneous recurrence relation of degree one. The recurrence relation f n = f n-1 + f n-2 is a linear homogeneous recurrence relation of degree two. The recurrence relation a n = a n-5 is a linear homogeneous recurrence relation of degree five. a n  2a n-1 + 1 is not homogeneous! a n = a n-1 + a n-2 2 is not linear !

32. 6.Advanced Counting Techniques 32 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations When solving such recurrence relations, we try basically to find solutions of the form a n = r n, where r is a constant. a n = r n is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 + … + c k a n-k if and only if r n = c 1 r n-1 + c 2 r n-2 + … + c k r n-k. Divide this equation by r n-k and subtract the right-hand side from the left: r k - c 1 r k-1 - c 2 r k-2 - … - c k-1 r - c k = 0 This is called the characteristic equation of the recurrence relation.

33. 6.Advanced Counting Techniques 33 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations The solutions of this equation are called the characteristic roots of the recurrence relation. The characteristic roots help in giving an explicit formula for the sequence. Let us consider linear homogeneous recurrence relations of degree two. Theorem 1: Let c 1 and c 2 be real numbers. Suppose that r 2 – c 1 r – c 2 = 0 has two distinct roots r 1 and r 2. Then the sequence {a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 if and only if a n =  1 r 1 n +  2 r 2 n for n = 0, 1, 2, …, where  1 and  2 are constants. See pp. 414 and 415 for the proof.

34. 6.Advanced Counting Techniques 34 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Example: What is the solution of the recurrence relation a n = a n-1 + 2a n-2 with a 0 = 2 and a 1 = 7 ? Solution: The characteristic equation of the recurrence relation is r 2 – r – 2 = 0. Its roots are r = 2 and r = -1. Hence, the sequence {a n } is a solution to the recurrence relation if and only if: a n =  1 2 n +  2 (-1) n for some constants  1 and  2.

35. 6.Advanced Counting Techniques 35 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Given the equation a n =  1 2 n +  2 (-1) n and the initial conditions a 0 = 2 and a 1 = 7, it follows that a 0 = 2 =  1 +  2 a 1 = 7 =  1  2 +  2  (-1) Solving these two equations gives us  1 = 3 and  2 = -1. Therefore, the solution to the recurrence relation and initial conditions is the sequence {a n } with a n = 3  2 n – (-1) n.

36. 6.Advanced Counting Techniques 36 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Example: What is the solution of the recurrence relation a n = a n-1 + 6a n-2 with a 0 = 1 and a 1 = 2 ? Solution: The characteristic equation of the recurrence relation is r 2 – r – 6 = 0. Its roots are r = 3 and r = -2. Hence, the sequence {a n } is a solution to the recurrence relation if and only if: a n =  1 3 n +  2 (-2) n for some constants  1 and  2.

37. 6.Advanced Counting Techniques 37 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Given the equation a n =  1 3 n +  2 (-2) n and the initial conditions a 0 = 1 and a 1 = 2, it follows that a 0 = 1 =  1 +  2 a 1 = 2 =  1  3 +  2  (-2) Solving these two equations gives us  1 = 4/5 and  2 = 1/5. Therefore, the solution to the recurrence relation and initial conditions is the sequence {a n } with a n = (4/5)(3) n + (1/5)(-2) n.

38. 6.Advanced Counting Techniques 38 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Example: Give an explicit formula for the Fibonacci numbers. Solution: The Fibonacci numbers satisfy the recurrence relation f n = f n-1 + f n-2 with initial conditions f 0 = 0 and f 1 = 1. The characteristic equation is r 2 – r – 1 = 0. Its roots are

39. 6.Advanced Counting Techniques 39 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations Therefore, the Fibonacci numbers are given by for some constants  1 and  2. We can determine values for these constants so that the sequence meets the conditions f 0 = 0 and f 1 = 1:

40. 6.Advanced Counting Techniques 40 Copyright M.R.K. Krishna Rao 2003 Solving Recurrence Relations The unique solution to this system of two equations and two variables is So finally we obtained an explicit formula for the Fibonacci numbers:

41. 6.Advanced Counting Techniques 41 Copyright M.R.K. Krishna Rao 2003 Homogeneous - Complications  Repeating roots in characteristic equation. Repeating roots imply that don’t learn anything new from second root, so may not have enough information to solve formula with given initial conditions. We’ll see how to deal with this on next slide.  Non-real number roots in characteristic equation. If the sequence has periodic behavior, may get complex roots (for example a n = -a n-2 ). We won’t worry about this case (in principle, same method works as before, except use complex arithmetic).

42. 6.Advanced Counting Techniques 42 Copyright M.R.K. Krishna Rao 2003 Complication: Repeating Roots EG: Solve a n = 2a n-1 -a n-2, a 0 = 1, a 1 = 2 Find characteristic equation by plugging in a n = r n : r 2 - 2r +1 = 0 Since r 2 - 2r +1 = (r -1) 2 the root r = 1 repeats. If we tried to solve by using general solution a n =  1 r 1 n +  2 r 2 n =  1 1 n +  2 1 n =  1 +  2 which forces a n to be a constant function (  ). SOLUTION: Multiply second solution by n so general solution looks like: a n =  1 r 1 n +  2 nr 1 n

43. 6.Advanced Counting Techniques 43 Copyright M.R.K. Krishna Rao 2003 Complication: Repeating Roots Solve a n = 2a n-1 -a n-2, a 0 = 1, a 1 = 2 General solution: a n =  1 1 n +  2 n1 n =  1 +  2 n Plug into initial conditions 1 = a 0 =  1 +  2.0·1 0 =  1 2 = a 1 =  1 ·1 1 +  2 ·1·1 1 =  1 +  2 Plugging first equation  1 = 1 into second: 2 = 1+  2 implies  2 = 1. Final answer: a n = 1+n

44. 6.Advanced Counting Techniques 44 Copyright M.R.K. Krishna Rao 2003 Complication: Repeating Roots In general, if the characteristic equation has only one root, the following theorem helps! Theorem 2: Let c 1 and c 2 be real numbers with c 2  0. Suppose that r 2 – c 1 r – c 2 = 0 has only one root r 0. A sequence {a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 if and only if a n =  1 r 0 n +  2 nr 0 n, for n = 0, 1, 2, …, where  1 and  2 are constants.

45. 6.Advanced Counting Techniques 45 Copyright M.R.K. Krishna Rao 2003 Complication: Repeating Roots Example: What is the solution of the recurrence relation a n = 6a n-1 – 9a n-2 with a 0 = 1 and a 1 = 6? Solution: The only root of r 2 – 6r + 9 = 0 is r 0 = 3. Hence, the solution to the recurrence relation is a n =  1 3 n +  2 n3 n for some constants  1 and  2. To match the initial condition, we need a 0 = 1 =  1 a 1 = 6 =  1  3 +  2  3 Solving these equations yields  1 = 1 and  2 = 1. Consequently, the overall solution is given by a n = 3 n + n3 n.

46. 6.Advanced Counting Techniques 46 Copyright M.R.K. Krishna Rao 2003 k-LiHoReCoCos Consider a k-LiHoReCoCo: It’s C.E. is: Thm.3: If this has k distinct roots r i, then the solutions to the recurrence are of the form: for all n≥0, where the α i are constants. Example 6 from textbook.

47. 6.Advanced Counting Techniques 47 Copyright M.R.K. Krishna Rao 2003 Degenerate k-LiHoReCoCos Theorem 4: Suppose there are t roots r 1,…,r t with multiplicities m 1,…,m t. Then: for all n≥0, where all the α are constants. Example 8 from textbook. Blackboard Ex. 18, 20, 22

48. 6.Advanced Counting Techniques 48 Copyright M.R.K. Krishna Rao 2003 The Nonhomogeneous Case Linear nonhomogeneous recurrences with constant coefficients may (unlike LiHoReCoCos) contain terms F(n) that depend only on n. Consider the Tower of Hanoi recurrence a n = 2a n-1 +1. Could solve using telescoping. Instead let’s solve it methodically. Rewrite: a n - 2a n-1 = 1  Solve with the RHS set to 0, i.e. solve the homogeneous case.  Add a particular solution to get general solution.

49. 6.Advanced Counting Techniques 49 Copyright M.R.K. Krishna Rao 2003 The Nonhomogeneous Case Add a particular solution to get general solution. I.e. use rule: a n - 2a n-1 = 1  Solve with the RHS set to 0, i.e. solve a n - 2a n-1 = 0 Characteristic equation: r - 2 = 0 so unique root is r = 2. General solution to homogeneous equation is a n =  1 2 n General Nonhomogeneous = General homogeneous Particular Nonhomogeneous +

50. 6.Advanced Counting Techniques 50 Copyright M.R.K. Krishna Rao 2003 The Nonhomogeneous Case  Add a particular solution to get general solution for a n - 2a n-1 = 1. Use rule:  There are little tricks for guessing particular nonhomogeneous solutions. For example, when the RHS is constant, the guess should also be a constant. So guess a particular solution of the form b n =C. Plug into the original recursion: 1 = b n – 2b n-1 = C – 2C = -C. Therefore C = -1. General solution: a n =  1 2 n -1. General Nonhomogeneous = General homogeneous Particular Nonhomogeneous +

51. 6.Advanced Counting Techniques 51 Copyright M.R.K. Krishna Rao 2003 The Non-homogeneous Case Finally, use initial conditions to get closed solution. In the case of the Towers of Hanoi recursion, initial condition is: a 1 = 1 Using general solution a n =  1 2 n -1 we get: 1 = a 1 =  1 2 1 -1 = 2  1 –1. Therefore, 2 = 2  1, so  1 = 1. Final answer: a n = 2 n -1

52. 6.Advanced Counting Techniques 52 Copyright M.R.K. Krishna Rao 2003 More Complicated Example: Find the general solution to recurrence from the bit strings example: a n = 2a n-2 + 2 n-3 - a n-3  Rewrite as a n - 2a n-2 + a n-3 = 2 n-3 and solve homogeneous part:  Characteristic equation: r 3 - 2r +1 = 0.  Guess root r =  1 as integer roots.  r = 1 works, so divide out by (r -1) to get  r 3 - 2r +1 = (r -1)(r 2 +r -1).

53. 6.Advanced Counting Techniques 53 Copyright M.R.K. Krishna Rao 2003 More Complicated r 3 - 2r +1 = (r -1)(r 2 +r -1). Quadratic formula on r 2 +r -1: r = (-1   5)/2 So r 1 = 1, r 2 = (-1+  5)/2, r 3 = (-1-  5)/2 General homogeneous solution: a n =  1 +  2 [(-1+  5)/2] n +  3 [(-1-  5)/2] n

54. 6.Advanced Counting Techniques 54 Copyright M.R.K. Krishna Rao 2003 More Complicated  Nonhomogeneous particular solution to  a n - 2a n-2 + a n-3 = 2 n-3  Guess the form b n = k 2 n. Plug guess in:  k 2 n - 2k 2 n-2 + k 2 n-3 = 2 n-3  Simplifies to: k =1/5.  So particular solution is b n = 2 n /5  Final (nearly) answer:  a n =  1 +  2 [(-1+  5)/2] n +  3 [(-1-  5)/2] n + 2 n /5 General Nonhomogeneous = General homogeneous Particular Nonhomogeneous +

55. 6.Advanced Counting Techniques 55 Copyright M.R.K. Krishna Rao 2003 Particular solutions table 1. Linear combinations work. 2. If f(n) contains r n and r is a characteristic root of multiplicity k, then multiply by n k.

56. 6.Advanced Counting Techniques 56 Copyright M.R.K. Krishna Rao 2003 An example from another textbook Blackboard Exercises: 32, 34