2. 15-251 Great Theoretical Ideas in Computer Science

3. X1 X1 X2 X2 + + X3 X3 Lecture 8 (September 17, 2009) Counting III: Generating Functions

4. Recap

5. The Binomial Formula (1+X) 0 = (1+X) 1 = (1+X) 2 = (1+X) 3 = (1+X) 4 = 1 1 + 1X 1 + 2X + 1X 2 1 + 3X + 3X 2 + 1X 3 1 + 4X + 6X 2 + 4X 3 + 1X 4 Pascal’s Triangle: k th row are coefficients of (1+X) k Inductive definition of k th entry of n th row: Pascal(n,0) = Pascal (n,n) = 1; Pascal(n,k) = Pascal(n-1,k-1) + Pascal(n-1,k)

6. Pascal’s Triangle “It is extraordinary how fertile in properties the triangle is. Everyone can try his hand” 11 121 1331 14641 15101051 1615201561

7. 11 121 1331 14641 15101051 1615201561 Summing on k th Avenue  i = k n i k n+1 k+1 =

8. 11 121 1331 14641 15101051 1615201561 Fibonacci Numbers = 2 = 3 = 5 = 8 = 13

9. 11 121 1331 14641 15101051 1615201561 Sums of Squares 222 2222 2n n n k  k = 0 n 2 =

10. All these properties can be proved inductively and algebraically. We gave combinatorial proofs using the Manhattan block walking representation of binomial coefficients

11. How many shortest routes from A to B? B A 10 5

12. Manhattan j th streetk th avenue 1 0 2 4 3 0 1 2 3 4 There areshortest routes from (0,0) to (j,k) j+k k

13. Manhattan Level nk th avenue 1 0 2 4 3 0 1 2 3 4 There areshortest routes from (0,0) to (n-k,k) n k

14. Manhattan Level nk th avenue 1 0 2 4 3 0 1 2 3 4 There are shortest routes from (0,0) to n k level n and k th avenue

15. Level nk th avenue 1 0 2 4 3 0 1 2 3 4 n k n-1 k-1 n-1 k = +

16. Level nk th avenue 1 0 2 4 3 0 1 2 3 4 n k 2 How many ways to get to via ?

17. Level nk th avenue 1 0 2 4 3 0 1 2 3 4 2n n n k  k = 0 n 2 =

18. Level nk th avenue 1 0 2 4 3 0 1 2 3 4 How many ways to get to via ? n+1 k+1 i k

19. Level nk th avenue 1 0 2 4 3 0 1 2 3 4 n+1 k+1 i k  i = k n = n+1 k+1

20. 1 + X 1 + X 2 + X 3 + … + X n-2 + X n-1 = X - 1 X n – 1 Recall the Geometric Series 1 - X 1 - X n =

21. 1 + X 1 + X 2 + X 3 + … + X n + … = 1 - X 1 the Infinite Geometric Series Also makes sense if we view the infinite sum on the left as a formal power series Holds when |X| < 1

22. 1 + X 1 + X 2 + X 3 + … + X n + … 1 - X 1 - X 1 - X 2 - X 3 - … - X n - X n+1 - … (1- X) P(X) =  P(X) = -X * P(X) = P(X) = 1

23. Formal Power Series  i = 0  aiXiaiXi P(X) = There are no worries about convergence issues. This is a purely syntactic object.

24. Formal Power Series  i = 0  aiXiaiXi P(X) = If you want, think of as the infinite vector V = But, as you will see, thinking of as a “polynomial” is very natural.

25. Operations A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … adding them together (A+B)(X) = (a 0 +b 0 ) + (a 1 +b 1 ) X + (a 2 +b 2 ) X 2 + … like adding the vectors position-wise + =

26. Operations A(X) = a 0 X 0 + a 1 X 1 + a 2 X 2 + … multiplying by X X * A(X) = 0 X 0 + a 0 X 1 + a 1 X 2 + a 2 X 3 + … like shifting the vector entries SHIFT =

27. Example Example: V = n th row of Pascal’s triangle Store: V = V := ; Loop n times V := V + SHIFT(V);

28. Example: V := ; Loop n times V := V + SHIFT(V); V = n th row of Pascal’s triangle P V := 1; P V := P V (1+X); Example

29. Example: V := ; Loop n times V := V + SHIFT(V); V = n th row of Pascal’s triangle Example P V = (1+ X) n As expected, the coefficients of P V give the n th row of Pascal’s triangle

30. To summarize…  i = 0  aiXiaiXi P(X) = Given a sequence V = associate a formal power series with it This is the “generating function” for V

31. Fibonaccis i.e., the sequence is represented by the power series 0 + 1X 1 + 1X 2 + 2 X 3 + 3 X 4 + 5 X 5 + 8 X 6 +… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2

32. Two Representations A(X) = 0 + 1X 1 + 1X 2 + 2 X 3 + 3 X 4 + 5 X 5 + 8 X 6 +… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 Can we write A(X) more succinctly?

33. A(X) = F 0 + F 1 X 1 + F 2 X 2 + F 3 X 3 + … + F n X n +… 0 + F 0 X 1 + F 1 X 2 + F 2 X 3 + … + F n-1 X n +… X A(X) = 0 + 0 X 1 + F 0 X 2 + F 1 X 3 + … + F n-2 X n +… X 2 A(X) = (1 – X – X 2 ) A(X) (F 0 – 0 – 0) + (F 1 – F 0 – 0)X 1 + 0 = X = (1 – X – X 2 ) X A(X) =

34. Fibonaccis F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 (1 – X – X 2 ) X A(X) = has the generating function i.e., the coefficient of X n in A(X) is F n

35. 1 – X – X 2 X X 2 + X 3 -(X – X 2 – X 3 ) X 2X 3 + X 4 -(X 2 – X 3 – X 4 ) + X 2 -(2X 3 – 2X 4 – 2X 5 ) + 2X 3 3X 4 + 2X 5 + 3X 4 -(3X 4 – 3X 5 – 3X 6 ) 5X 5 + 3X 6 + 5X 5 -(5X 5 – 5X 6 – 5X 7 ) 8X 6 + 5X 7 + 8X 6 -(8X 6 – 8X 7 – 8X 8 )

36. Two representations of the same thing… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 (1 – X – X 2 ) X A(X) =

37. Closed form? F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 (1 – X – X 2 ) X A(X) = let’s factor (1 – X – X 2 ) (1 – X – X 2 ) = (1 - φ X)(1 – (-1/ φ )X) where φ = 1 + √5 2

38. Simplify, simplify… F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 X A(X) = (1 - φ X)(1 – (-1/ φ )X) A(X) = 1 (1 - φ X) 1 (1 – (-1/ φ )X) √5 1 + some elementary algebra omitted…* *you are not allowed to say this in your answers…

39. A(X) = 1 (1 - φ X) 1 (1 – (-1/ φ )X) √5 1 + 1 (1 - φ X) = 1 + φ X + φ 2 X 2 + … + φ n X n + … = 1 + Y 1 + Y 2 + Y 3 + … + Y n + … 1 - Y 1 the Infinite Geometric Series

40. A(X) = 1 (1 - φ X) 1 (1 – (-1/ φ )X) √5 1 + 1 (1 - φ X) = 1 + φ X + φ 2 X 2 + … + φ n X n + …  the coefficient of X n in A(X) is… √5 1 (-1/ φ ) n √5 + φnφn 1 (1 – (-1/ φ )X) = 1 + (-1/ φ ) X + … + (-1/ φ ) n X n + …

41. √5 1 (-1/ φ ) n √5 + φnφn F n = where φ = 1 + √5 2 J.P.M. Binet (1843) Closed form for Fibonaccis Leonhard Euler (1765) Abraham de Moivre (1730) “golden ratio”

42. √5 1 (-1/ φ ) n √5 + φnφn F n = Closed form for Fibonaccis √5 1 φnφn F n = closest integer to

44. To recap…  i = 0  aiXiaiXi P(X) = Given a sequence V = associate a formal power series with it This is the “generating function” for V We just used this for the Fibonaccis…

45. Multiplication A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … multiply them together (A*B)(X) = (a 0* b 0 ) + (a 0 b 1 + a 1 b 0 ) X + (a 0 b 2 + a 1 b 1 + a 2 b 0 ) X 2 + (a 0 b 3 + a 1 b 2 + a 2 b 1 + a 3 b 0 ) X 3 + … seems a bit less natural in the vector representation (it’s called a “convolution” there)

46. Mult.: special case A(X) = a 0 + a 1 X + a 2 X 2 + … Special case: B(X) = 1 + X + X 2 + … multiply them together (A*B)(X) = a 0 + (a 0 + a 1 ) X + (a 0 + a 1 + a 2 ) X 2 + (a 0 + a 1 + a 2 + a 3 ) X 3 + … it gives us partial sums! 1 1-X =

47. For example… Suppose A(X) = 1 + X + X 2 + … and B(X) = 1 + X + X 2 + … 1 1-X = 1 = then (A*B)(X) = 1 + 2X + 3X 2 + 4X 3 + … 1 1-X = 1 * 1 (1-X) 2 = Generating function for the sequence

48. Take 1 + 2X + 3X 2 + 4X 3 + … 1 (1-X) 2 = ¢ 1 + ¢ 2 X 1 + ¢ 3 X 2 + ¢ 4 X 3 +… 1 (1-X) 3 = multiplying through by 1/(1-X) Generating function for the triangular numbers! What happens if we again take prefix sums?

49. 1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = What’s the pattern? 1 (1-X) n = ???

50. 1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = 0 0 1 0 2 0 3 0 1 (1-X) n = ??? What’s the pattern?,,,, …

51. 1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = 0 0 1 0 2 0 3 0 1 1 2 1 3 1 4 1,,,, … 1 (1-X) n = ??? What’s the pattern?

52. 1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = 0 0 1 0 2 0 3 0 1 1 2 1 3 1 4 1,,,, … 2 2 3 2 4 2 5 2 1 (1-X) n = ??? What’s the pattern?

53. 1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = 0 0 1 0 2 0 3 0 1 1 2 1 3 1 4 1,,,, … 2 2 3 2 4 2 5 2 1 (1-X) n = What’s the pattern?

54. 1 (1-X) 2 = 1 (1-X) 3 = 1 1-X = 0 0 1 0 2 0 3 0 1 1 2 1 3 1 4 1,,,, … 2 2 3 2 4 2 5 2 1 (1-X) n =  k = 0  XkXk k+n-1 n-1 What’s the pattern?

55. 11 121 1331 14641 15101051 1615201561 From Lecture #7  i = 0 k i n k+1 n+1 = Can be used to derive the coefficient of X k

56. 1 (1-X) n =  k = 0  XkXk k+n-1 n-1 What’s the pattern?

57. What is the coefficient of X k in the expansion of: ( 1 + X + X 2 + X 3 + X 4 +.... ) n ? Each path in the choice tree for the cross terms has n choices of exponent e 1, e 2,..., e n ¸ 0. Each exponent can be any natural number. Coefficient of X k is the number of non-negative solutions to: e 1 + e 2 +... + e n = k Another way to see it…

58. n + k - 1 n - 1 What is the coefficient of X k in the expansion of: ( 1 + X + X 2 + X 3 + X 4 +.... ) n ? Another way to see it…

59. Recap: getting partial sums A(X) = a 0 + a 1 X + a 2 X 2 + … a 0 + (a 0 + a 1 ) X + (a 0 + a 1 + a 2 ) X 2 + (a 0 + a 1 + a 2 + a 3 ) X 3 + … dividing by (1-X) gives us partial sums! A(X) 1-X =

60. Here’s an interesting use… 1 (1-X) 3 =  k = 0  XkXk (k+2)(k+1) 2 subtract off and add k+2 2 3/2 (1-X) 2 1 1-X ½ X(1+X) 2(1-X) 3 =  k = 0  XkXk k2k2 2

61. From the previous page… X(1+X) (1-X) 3 =  k = 0  XkXk sum of first k squares hence if we take partial sums… X(1+X) (1-X) 3 = 1 1-X *  k = 0  XkXk k2k2

62. Coefficient of X k in (X 2 +X)/(1-X) 4 is the sum of the first k squares:

63. So finally…

64. Finally, a different counting problem…

65. Let c n = number of ways to pick exactly n fruits. What is a closed form for c n ? E.g., c 5 = 6

66. Recall Multiplication A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … multiply them together (A*B)(X) = (a 0* b 0 ) + (a 0 b 1 + a 1 b 0 ) X + (a 0 b 2 + a 1 b 1 + a 2 b 0 ) X 2 + (a 0 b 3 + a 1 b 2 + a 2 b 1 + a 3 b 0 ) X 3 + …

67. So if A(x), B(x), O(x) and P(x) are the generating functions for the number of ways to fill baskets using only one kind of fruit the generating function for number of ways to fill basket using any of these fruit is given by C(x) = A(x)B(x)O(x)P(x)

68. Suppose we only pick bananas b n = number of ways to pick n fruits, only bananas. B(x) = 1 + x 2 + x 4 + x 6 + … 1 1-X 2 =

69. Suppose we only pick apples a n = number of ways to pick n fruits, only apples. A(x) = 1 + x 5 + x 10 + x 15 + … 1 1-X 5 =

70. Suppose we only pick oranges o n = number of ways to pick n fruits, only oranges. O(x) = 1 + x + x 2 + x 3 + x 4 1-X 5 1-X =

71. Suppose we only pick pears p n = number of ways to pick n fruits, only pears. P(x) = 1 + x 1-X 2 1-X =

72. Let c n = number of ways to pick exactly n fruits of any type  c n x n = A(x) B(x) O(x) P(x) = 1-X 2 1-X 1 1-X 5 1 1-X 2 1-X 5 1-X = 1 (1-X) 2

73. 1 c n is coefficient of X n in c n = n+1. Let c n = number of ways to pick exactly n fruits of any type

74. More operations: Differentiation A(X) = a 0 + a 1 X + a 2 X 2 + … differentiate it… A’(X) = a 1 + 2a 2 X + 3a 3 X 2 …  i = 0  (i+1)a i+1 X i A’(X) =  i = 0  ia i X i X A’(X) =

75. Formal Power Series Basic operations on Formal Power Series Writing the generating function for a recurrence binomial and multinomial coefficients Solving G.F. to get closed form G.F.s for common sequences Prefix sums using G.F.s Using G.F.s to solve counting problems Here’s What You Need to Know…