1. Induction: One Step At A Time
Great Theoretical Ideas In Computer Science
Steven Rudich
CS Spring 2003
Lecture 3
Jan 21, 2003
Carnegie Mellon University
Induction: One Step At A Time

2. Last time we talked about different ways to represent numbers: unary, binary, decimal, base b, plus/minus binary, Egyptian binary . . .

3. Different representations had different advantages and disadvantages.

4. Today we will talk about INDUCTION

5. Induction is the primary way we:
Prove theorems
Construct and define objects

6. Representing a problem or object inductively is one of the most fundamental abstract representations.

7. Let’s start with dominoes

8. Domino Principle: Line up any number of dominos in a row; knock the first one over and they will all fall.

9. n dominoes numbered 1 to n
Fk ´ The kth domino will fall
If we set them all up in a row then we know that each one is set up to knock over the next one:
For all 1· k < n:
Fk ) Fk+1

10. n dominoes numbered 1 to n
Fk ´ The kth domino will fall
For all 1· k < n:
Fk ) Fk+1
F1 ) F2 ) F3 ) …
F1 ) All Dominoes Fall

11. n dominoes numbered 0 to n-1
Fk ´ The kth domino will fall
For all 0· k < n-1:
Fk ) Fk+1
F0 ) F1 ) F2 ) …
F0 ) All Dominoes Fall

12. The Natural Numbers
 = { 0, 1, 2, 3, . . .}

13. Plato: The Domino Principle works for an infinite row of dominoes
Aristotle: An “infinite” row? What do you mean? Can there be such a thing?

14. The Infinite Domino Principle
Fk ´ The kth domino will fall Assume we know that for every natural number k, Fk ) Fk+1
F0 ) F1 ) F2 ) …
F0 ) All Dominoes Fall

15. Plato’s Dominoes One for each natural number
An infinite row, 0, 1, 2, … of dominoes, one domino for each natural number. Knock the first domino over and they all will fall.
Proof: Suppose they don’t all fall. Let k>0 be the lowest numbered domino that remains standing. Domino k-1¸0 did fall, but k-1 will knock over domino k. Thus, domino k must fall and remain standing. Contradiction.

16. The Infinite Domino Principle
Fk ´ The kth domino will fall Assume we know that for every natural number k, Fk ) Fk+1
F0 ) F1 ) F2 ) …
F0 ) All Dominoes Fall

17. Mathematical Induction: statements proved instead of dominoes fallen
Infinite sequence of
dominoes.
Fk ´ domino k fell
Infinite sequence of statements: S0, S1, …
Fk ´ Sk proved
Establish 1) F0
2) For all k, Fk ) Fk+1
Conclude that Fk is true for all k

18. Inductive Proof / Reasoning To Prove k, Sk
Establish “Base Case”: S0
Establish that k, Sk ) Sk+1
Assume hypothetically that
Sk for any particular k;
Conclude that Sk+1
k, Sk ) Sk+1

19. Inductive Proof / Reasoning To Prove k, Sk
Establish “Base Case”: S0
Establish that k, Sk ) Sk+1
Assume “Inductive Hypothesis”: Sk
Prove that Sk+1 follows from Sk

20. Inductive Proof / Reasoning To Prove k¸b, Sk
Establish “Base Case”: Sb
Establish that k¸b, Sk ) Sk+1
Assume k¸ b
Assume “Inductive Hypothesis”: Sk
Prove that Sk+1 follows

21. We already know that n, n= 1 + 2 + 3 + . . . + n-1 + n = n(n+1)/2.
Let’s prove it by induction:
Let Sn ´
“n =n(n+1)/2”

22. Sn ´ “n =n(n+1)/2” Use induction to prove k¸0, Sk
Establish “Base Case”: S0. 0=The sum of the first 0 numbers = 0. Setting n=0 the formula gives 0(0+1)/2 = 0.
Establish that k¸0, Sk ) Sk+1
“Inductive Hypothesis” Sk: k =k(k+1)/2
k+1 = k (k+1) = k(k+1)/2 + (k+1) [Using I.H.]
= (k+1)(k+2)/ [which proves Sk+1]

23. Aristotle’s Contrapositive
Let S be a sentence of the form “A ) B”.
The Contrapositive of S is the sentence
“B ) A”.
The two sentences are logically equivalent. That is, one is true in exactly the same situations where the other is true.
Hence, one can just as well prove the Contrapositive of S to establish S.

24. Contrapositive Dominos
Suppose there is a least domino k that does not fall.
If k did not fall, then k-1 did not fall. k-1 would be a smaller, standing domino, contradicting our assumption.

25. Contrapositive or Least Counter-Example Induction: to Prove k, Sk
Establish “Base Case”: S0
Establish that k, Sk ) Sk+1
Let k>0 be the least number such that Sk is false.
Prove that Sk ) Sk-1
[Contradiction of k being
the least counter-example]

26. “Strong” Induction To Prove k, Sk
Establish “Base Case”: S0
Establish that k, Sk ) Sk+1
Let k be any natural number.
Assume j<k, Sj
Prove Sk

27. Strong, Contrapositive Induction:
Assume there is a least counter-example. Derive the existence of a smaller counter-example. Conclude there is no counter-example.

28. Euclid’s theorem on the unique factorization of a number into primes.
Assume there is a least counter-example. Derive the existence of a smaller counter-example.

29. Theorem: Each natural has a unique factorization into primes written in non-decreasing order.
Definition: A number > 1 is prime if it has no other factors, besides 1 and itself.
Primes: 2, 3, 5, 7, 11, 13, 17, ….
Factorizations:
42 = 2 * 3 * 7
84 = 2 * 2 * 3 * 7
= 13

30. Theorem: Each natural has a unique factorization into primes written in non-decreasing order.
Let n be the least counter-example. n has at least two ways of being written as a product of primes:
n = p1 p2 .. pk = q1 q2 … qt
The p’s must be totally different primes than the q’s or else we could divide both sides by one of a common prime and get a smaller counter-example. Without loss of generality, assume q1 < p1 .

31. Theorem: Each natural has a unique factorization into primes written in non-decreasing order.
Let n be the least counter-example.
n = p1 p2 .. pk = q1 q2 … qt [ q1 < p1 ]
n ¸ (p1)2 > p1 q1 + 1
m = n – p1q1 < n
Both p1 and q1 divide n, and hence divide m (so m>1)
By unique factorization of m, p1q1 divides m = p1q1z
m = n – p1q1 = p1(p2 .. pk - q1 ) = p1q1z
(p2 .. pk - q1 ) = q1z
p2 .. pk = q1z + q1 = q1(z+1) so q1 divides p2…pk
And hence, by unique factorization of p2…pk,
q1 must be one of the primes p2,…,pk. Contradiction of q1<p1.

32. Inductive Reasoning is the high level idea:
“Standard” Induction
“Contrapositive” Induction
“Strong” Induction are just different packaging.

33. “Strong” Induction Can Be Repackaged As Standard Induction
Define Ti = j· i, Sj
Establish “Base Case”: T0
Establish that k, Tk ) Tk+1
Let k be any natural number.
Assume Tk-1
Prove Tk
Establish “Base Case”: S0
Establish that k, Sk ) Sk+1
Let k be any natural number.
Assume j<k, Sj
Prove Sk

34. Yet another way of packaging inductive reasoning is to define an “invariant”.
Not varying; constant.
Mathematics. Unaffected by a designated operation, as a transformation of coordinates.

35. Yet another way of packaging inductive reasoning is to define an “invariant”.
Invariant: 3. programming A rule, such as the ordering an ordered list or heap, that applies throughout the life of a data structure or procedure. Each change to the data structure must maintain the correctness of the invariant.

36. Invariant Induction Suppose we have a time varying world state: W0, W1, W2, … Each state change is assumed to come from a list of permissible operations. We seek to prove that statement S is true of all future worlds.
Argue that S is true of the initial world.
Show that if S is true of some world – then S remains true after one permissible operation is performed.

37. Invariant Induction Suppose we have a time varying world state: W0, W1, W2, … Each state change is assumed to come from a list of permissible operations.
Let S be a statement true of W0.
Let W be any possible future world state.
Assume S is true of W.
Show that S is true of any world W’ obtained by applying a permissible operation to W.

38. Odd/Even Handshaking Theorem: At any party at any point in time define a person’s parity as ODD/EVEN according to the number of hands they have shaken. Statement: The number of people of odd parity must be even.
Zero hands have been shaken at the start of a party, so zero people have odd parity. If 2 people of different parities shake, then they both swap parities and the odd parity count is unchanged. If 2 people of the same parity shake, they both change and hence the odd parity count changes by 2 – and remains even.

39. Inductive Reasoning is the high level idea:
“Standard” Induction
“Contrapositive” Induction
“Strong” Induction
and Invariants are just different packaging.

40. Induction is also how we can define and construct our world.

41. So many things, from buildings to computers, are built up stage by stage, module by module, each depending on the previous stages.

42. Well, almost always

43. Inductive Definition Of Functions
Stage 0, Initial Condition, or Base Case:
Declare the value of the function on some subset of the domain.
Inductive Rules
Define new values of the function in terms of previously defined values of the function
F(x) is defined if and only if it is implied by finite iteration of the rules.

44. Inductive Definition Recurrence Relation for F(X)
Initial Condition, or Base Case:
F(0) = 1
Inductive Rule
For n>0, F(n) = F(n-1) + F(n-1) n 1 2 3 4 5 6 7
F(n) 8 16 32 64
128

45. Inductive Definition Recurrence Relation for F(X) = 2X
Initial Condition, or Base Case:
F(0) = 1
Inductive Rule
For n>0, F(n) = F(n-1) + F(n-1) n 1 2 3 4 5 6 7
F(n) 8 16 32 64
128

46. Inductive Definition Recurrence Relation
Initial Condition, or Base Case:
F(1) = 1
Inductive Rule
For n>1, F(n) = F(n/2) + F(n/2) n 1 2 3 4 5 6 7
F(n) %

47. Inductive Definition Recurrence Relation F(X) = X for X a whole power of 2.
Initial Condition, or Base Case:
F(1) = 1
Inductive Rule
For n>1, F(n) = F(n/2) + F(n/2) n 1 2 3 4 5 6 7
F(n) %

48. Leonardo Fibonacci
In 1202, Fibonacci proposed a problem about the growth of rabbit populations.

49. Leonardo Fibonacci
In 1202, Fibonacci proposed a problem about the growth of rabbit populations.

50. Leonardo Fibonacci
In 1202, Fibonacci proposed a problem about the growth of rabbit populations.

51. The rabbit reproduction model
A rabbit lives forever
The population starts as a single newborn pair
Every month, each productive pair begets a new pair which will become productive after 2 months old
Fn= # of rabbit pairs at the beginning of the nth month
month 1 2 3 4 5 6 7
rabbits 8 13

52. The rabbit reproduction model
A rabbit lives forever
The population starts as a single newborn pair
Every month, each productive pair begets a new pair which will become productive after 2 months old
Fn= # of rabbit pairs at the beginning of the nth month
month 1 2 3 4 5 6 7
rabbits 8 13

53. The rabbit reproduction model
A rabbit lives forever
The population starts as a single newborn pair
Every month, each productive pair begets a new pair which will become productive after 2 months old
Fn= # of rabbit pairs at the beginning of the nth month
month 1 2 3 4 5 6 7
rabbits 8 13

54. The rabbit reproduction model
A rabbit lives forever
The population starts as a single newborn pair
Every month, each productive pair begets a new pair which will become productive after 2 months old
Fn= # of rabbit pairs at the beginning of the nth month
month 1 2 3 4 5 6 7
rabbits 8 13

55. The rabbit reproduction model
A rabbit lives forever
The population starts as a single newborn pair
Every month, each productive pair begets a new pair which will become productive after 2 months old
Fn= # of rabbit pairs at the beginning of the nth month
month 1 2 3 4 5 6 7
rabbits 8 13

56. The rabbit reproduction model
A rabbit lives forever
The population starts as a single newborn pair
Every month, each productive pair begets a new pair which will become productive after 2 months old
Fn= # of rabbit pairs at the beginning of the nth month
month 1 2 3 4 5 6 7
rabbits 8 13

57. The rabbit reproduction model
A rabbit lives forever
The population starts as a single newborn pair
Every month, each productive pair begets a new pair which will become productive after 2 months old
Fn= # of rabbit pairs at the beginning of the nth month
month 1 2 3 4 5 6 7
rabbits 8 13

58. Inductive Definition or Recurrence Relation for the Fibonacci Numbers
Stage 0, Initial Condition, or Base Case:
Fib(1) = 1; Fib (2) = 1
Inductive Rule
For n>3, Fib(n) = Fib(n-1) + Fib(n-2) n 1 2 3 4 5 6 7
Fib(n) % 8 13

59. Inductive Definition or Recurrence Relation for the Fibonacci Numbers
Stage 0, Initial Condition, or Base Case:
Fib(0) = 0; Fib (1) = 1
Inductive Rule
For n>1, Fib(n) = Fib(n-1) + Fib(n-2) n 1 2 3 4 5 6 7
Fib(n) 8 13

60. An inductive definition/viewpoint on an object is one of many equivalent abstract representations of the object.

61. Inductive Definition of T(n)
T(n) = 4 T(n/2) + n
Notice that T(n) is inductively defined for positive powers of 2, and undefined on other values.

62. Closed Form Definition of G(n)
G(n) = 2n2 - n
Domain of G are the powers of two.

63. Two equivalent definitions?
G(n) = 2n2 - n
Domain of G are the powers of two.
T(1) = 1
T(n) = 4 T(n/2) + n

64. Prove equivalence by induction on n: Assume T(x) = G(x) for x < n
Closed Form: G(n) = 2n2 – n
Base: G(1) = 1 and T(1) = 1
Assuming T(n/2) = G(n/2) = 2(n/2)2 – n/2
T(n) = 4 T(n/2) + n
=4[G(n/2) + n]
4 [2(n/2)2 – n/2] + n
= 2n2 – 2n n
= 2n2 – n
= G(n)

65. Solving Recurrences Guess and Verify
Guess: G(n) = 2n2 – n
Verify: G(1) = 1 and G(n) = 4 G(n/2) + n
Similarly:T(1) = 1 and T(n) = 4 T(n/2) + n
So T(n) = G(n)

66. Non-Inductive Definition
Let Tilen be the number of different ways to tile a 1 X n strip with squares and dominoes.

67. Each tiling of the n-strip starts with either a square or a domino:
Inductive Insight
Let Tilen be the number of different ways to tile a 1 X n strip with squares and dominoes.
Each tiling of the n-strip starts with either a square or a domino:

68. Tilen = Tilen-1 + Tilen-2
Let Tilen be the number of different ways to tile a 1 X n strip with squares and dominoes.
Tilen-1 with initial square
Tilen-2 with initial domino

69. Two Equivalent Definitions
Non-inductive: Tilen is the number of ways to tile a 1 by n strip with squares and dominoes.
Inductive:
Tile1 = 1, Tile2 = 2
Tilen = Tilen-1 + Tilen-2
Equivalence is proven by induction on n

70. Proven by induction on n
Tilen = Fib(n+1)
Inductive:
Fib(1) = 1; Fib(2) = 1;
Fib(n) = Fib(n-1) + Fib(n-2)
Tile1 = 1, Tile2 = 2
Tilen = Tilen-1 + Tilen-2
Proven by induction on n

71. Another Ancient Definition Pascal(r,i)
Initial Condition, or Base Case:
Pascal(anything, negative number) = 0
Pascal(1,0) = 1
Inductive Rule
For i>0, Pascal(r,i) = Pascal(r-1,i-1) + Pascal(r,i)

72. Pascal’s Triangle
Al-Karaji, Baghdad
Chu Shin-Chieh 1303 The Precious Mirror of the Four Elements
. . . Known in Europe by 1529
Blaise Pascal 1654 1
1 1