CS121 Induction Alan J. Hu 1

Induction: The Big Picture We’ll develop your intuition about induction. We’ll see examples of weak, strong, and structural induction. Two different learning goals: 1.Learning the CONCEPT of induction. 2.Learning the FORM of an induction proof. We’ll see why they are really all the same.

A Little Game Are you all willing to stand up and move around the classroom? (If not, I can still convey the ideas, but it won’t be as memorable.)

A Little Game Everyone in this room fails CPSC 121, unless…

A Little Game Everyone in this room fails CPSC 121, unless… you make yourself “safe” by following these rules: 1. The podium at the front of class is safe. 2. Anyone who puts both hands on something or someone safe is also safe. (3. There is no other way to be safe.)

How do you make everyone safe? Podium

Do you have to fight for the podium? Podium

You can be good Canadians… Podium

You can be less structured… Podium

How many people can you make safe? Podium

Is this OK? Podium

Or worse, this? Podium

Who fails now? Podium

Who fails here? Podium

What’s the point? This is exactly how induction works in general! Base Case: Some cases where you know the property holds already. Inductive Case: You must show that the property holds for any given item, by showing how it is implied by the property on items closer to the base case. If you can show the inductive case for all elements in a set, the property holds for all of them – even an infinite number!

What’s the point? This is exactly how induction works in general! If anyone breaks the chain, the proof fails. If anyone misses the base case, the proof fails. If there are any cycles (rather than getting closer to base case), the proof fails. On the other hand, if you get induction right, it’s a very easy and powerful way to write a proof!

Weak Induction This is what you are usually taught first. Goal: Prove p(n) for all natural numbers n Base Case: Prove p(0) (and maybe p(1), etc.) Inductive Case: Assume p(k) for some arbitrary k and prove that it implies p(k+1).

Weak Induction: What’s really happening? This is what you are usually taught first. Goal: Prove p(n) for all natural numbers n Base Case: Prove p(0) (and maybe p(1), etc.) Inductive Case: Assume p(k) for some arbitrary k and prove that it implies p(k+1). This is just the polite Canadians version of the game! p(x) just means x is safe. Base case is the podium. Inductive case says that an arbitrary p(k) is true because he’s got both hands on p(k-1), who is closer to the base case.

Weak Induction: What your proof should look like Problem: Prove that the solution to the n disk Towers of Hanoi takes 2^n-1 moves. (Aside: Do you know this problem?)

Weak Induction: What your proof should look like Problem: Prove that the solution to the n disk Towers of Hanoi takes 2^n-1 moves. Proof: The proof is by induction on n. Base Case: n=0. In this case, 0 moves are required. 2^0 – 1 = 0, so the property holds on the base case.

Weak Induction: What your proof should look like Problem: Prove that the solution to the n disk Towers of Hanoi takes 2^n-1 moves. Proof: The proof is by induction on n. Inductive Case: The inductive assumption is that moving a stack of n-1 disks takes 2^(n-1) – 1 moves. Moving a stack of n disks means moving n-1 disks to the intermediate stack, moving the nth disk from start to destination, and the moving the n-1 disks from the intermediate stack to the destination. (continued)

Weak Induction: What your proof should look like Problem: Prove that the solution to the n disk Towers of Hanoi takes 2^n-1 moves. Proof: The proof is by induction on n. Inductive Case: (continued) Using the inductive assumption, the total number of moves is 2^(n-1) – ^(n-1) – 1 which simplifies to 2^n – 1. QED

Weak Induction: What your proof should look like Problem: Prove that the solution to the n disk Towers of Hanoi takes 2^n-1 moves. Proof: The proof is by induction on n. Inductive Case: (continued) Using the inductive assumption, the total number of moves is 2^(n-1) – ^(n-1) – 1 which simplifies to 2^n – 1. QED (Aside: This is usually taught as assuming n and proving n+1, which is OK, too.)

Strong Induction This is often taught as something special… Goal: Prove p(n) for all natural numbers n Base Case: Prove p(0) (and maybe p(1), etc.) Inductive Case: Assume p(0) through p(k) for some arbitrary k and prove that it implies p(k+1).

Strong Induction: What’s Really Happening This is often taught as something special… Goal: Prove p(n) for all natural numbers n Base Case: Prove p(0) (and maybe p(1), etc.) Inductive Case: Assume p(0) through p(k) for some arbitrary k and prove that it implies p(k+1). This is just the unstructured version of the game! As long as p(k+1) has as many hands as the rules require on p(0)…p(k) (who are closer to base case), the property holds.

Strong Induction: What your proof should look like Problem: Prove that the nth Fibonacci number is less than 2^n. (Aside: Do you know this problem?)

Strong Induction: What your proof should look like Problem: Prove that the nth Fibonacci number is less than 2^n. Proof: The proof is by (strong) induction on n. Base Case: When n=1, the 1 st Fibonacci number is 1, which is less than 2, which is 2^1. When n=2, the 2 nd Fibonacci number is also 1, which is less than 4, which is 2^2. So the base cases hold.

Strong Induction: What your proof should look like Problem: Prove that the nth Fibonacci number is less than 2^n. Proof: The proof is by (strong) induction on n. Inductive Case: For all n>2, the nth Fibonacci number is defined to be fib(n-1)+fib(n-2). By the inductive assumption, fib(n-1) < 2^(n-1) and fib(n-2) < 2^(n-2). Therefore, fib(n) = fib(n-1) + fib(n-2) < 2^(n-1) + 2^(n-2) < 2^(n-1) + 2^(n-1) = 2^n. QED

Fibonacci proof in pictures fib(1) < 2^1 fib(3)<2^3 fib(4)<2^4) fib(5)<2^5 fib(2) < 2^2 fib(6)<2^6) fib(7)<2^7

Strong Induction: What went wrong? Theorem: All horses are the same colour. Proof: By (strong) induction on n, the number of horses in any set of horses. Base Case: n=1. 1 horse is the same colour as itself. Inductive Case: Assume all sets of horses of size from 1 to n- 1 are the same colour. Given any set of horses of size n, separate one from the herd. Call this horse A. By the inductive assumption, the remaining n-1 horses are the same colour. Now, put back the first horse and separate out a different horse, called B; similarly, the remaining n-1 horses are the same colour.

Strong Induction: What went wrong? Theorem: All horses are the same colour. Proof: By (strong) induction on n, the number of horses in any set of horses. Base Case: n=1. 1 horse is the same colour as itself. Inductive Case: (continued) But, those two sets of n-1 horses had n-2 horses in common, and horses don’t change colour! So A has the same colour as the n-2, who have the same colour as B. So, they all must be the same colour, by transitivity of equality. QED

Structural Induction This is extremely common in computer science! Goal: Prove p(n) for all structures n. (Aside: This only works when the set of all possible structures is defined recursively/inductively.) Base Case: Prove p for all the base case structures. Inductive Case: For an arbitrary non-base-case structure n, show that p(n) holding on the structure is implied by p holding on the sub-structures.

Structural Induction: What’s Really Happening This is extremely common in computer science! Goal: Prove p(n) for all structures n. (Aside: This only works when the set of all possible structures is defined recursively/inductively.) Base Case: Prove p for all the base case structures. Inductive Case: For an arbitrary non-base-case structure n, show that p(n) holding on the structure is implied by p holding on the sub-structures. This is still the same game!

Structural Induction: What your proofs should look like Problem: Prove that a perfect/complete n-ary tree of height h has (n^(h+1) – 1)/(n – 1) nodes. Aside: Do you know what a tree is yet? A tree is either a leaf (a node with no children), or a node whose children are trees. In an n-ary tree, the each non-leaf node has n children. In a perfect/complete tree, all leaves are at the same depth.

Structural Induction: What your proofs should look like Problem: Prove that a perfect/complete n-ary tree of height h has (n^(h+1) – 1)/(n – 1) nodes. Aside: Do you know what a tree is yet? A tree is either a leaf (a node with no children), or a node whose children are trees. In an n-ary tree, the each non-leaf node has n children. In a perfect/complete tree, all leaves are at the same depth.

Structural Induction: What your proofs should look like Problem: Prove that a perfect/complete n-ary tree of height h has (n^(h+1) – 1)/(n – 1) nodes. Proof: The proof is a structural induction on trees. Base Case: If the tree is a single leaf, then it has height 0 and 1 node. The formula evaluates as (n^(0+1) – 1)/(n – 1) = 1.

Structural Induction: What your proofs should look like Problem: Prove that a perfect/complete n-ary tree of height h has (n^(h+1) – 1)/(n – 1) nodes. Proof: The proof is a structural induction on trees. Inductive Case: If the tree is not a leaf, then it must have n children (because it is n-ary), each of which is a tree. Because it is perfect/complete, the height of each of these subtrees is h-1. (continued)

Structural Induction: What your proofs should look like Problem: Prove that a perfect/complete n-ary tree of height h has (n^(h+1) – 1)/(n – 1) nodes. Proof: The proof is a structural induction on trees. Inductive Case: (continued) By the inductive hypothesis, each of those trees, therefore, have (n^((h-1)+1) – 1)/(n – 1) nodes. Adding up all the nodes, we have n times the above, plus 1 for the root. Simplifying the math, we have…

Structural Induction: What your proofs should look like QED!

Induction Is Induction Weak, strong, and structural induction get taught as if they were separate concepts, but they are all the same. We will see: –Weak and strong induction are the same. –Weak is a special case of structural induction.

Weak Induction = Strong Induction We are trying to prove some property P(n) for all natural numbers n. Base cases are the same: Prove P(0) or other small cases. In weak induction, we get to assume P(k-1) to prove P(k). In strong induction, we get to assume P(0) through P(k-1) to prove P(k). So, clearly, every weak induction proof is a strong induction proof. What about the other way around?

Weak Induction = Strong Induction We are trying to prove some property P(n) for all natural numbers n. But we have no limitations on what P(n) is! Given a strong induction proof for P(n), define Q(n) to be the property: Q(n) = “P(k) holds for all 0<=k<n” The strong induction proof of P(n) becomes a weak induction proof on Q(n).

Weak Induction Is Structural Induction Weak induction proves p(n) for all natural numbers. Structural induction proves p(n) for all structures n, which are defined inductively. What are natural numbers, really?

Peano Postulates/Axioms By Giuseppe Peano, in This is the formal way mathematicians define natural numbers.

Peano Postulates/Axioms 1.0 is a natural number. 2.For every natural number n, S(n) is a natural number. 3.For every natural number n, S(n)!=0. 4.For all natural numbers m and n, if S(m)=S(n), then m=n. 5.If K is any set that obeys #1 and #2, then K contains all natural numbers.

Peano Postulates/Axioms 1.0 is a natural number. 2.For every natural number n, S(n) is a natural number. 3.For every natural number n, S(n)!=0. 4.For all natural numbers m and n, if S(m)=S(n), then m=n. 5.If K is any set that obeys #1 and #2, then K contains all natural numbers. Natural numbers are defined inductively!

Peano Postulates/Axioms 1.0 is a natural number. 2.For every natural number n, S(n) is a natural number. 3.For every natural number n, S(n)!=0. 4.For all natural numbers m and n, if S(m)=S(n), then m=n. 5.If K is any set that obeys #1 and #2, then K contains all natural numbers. Nothing else is a natural number. (So, if you do the structural induction, you’ve proven your result for all n.)

Weak Induction Is Structural Induction Weak induction is just structural induction on the natural numbers (as they are formally defined). Base Case: Prove the property for p(0). Inductive Case: Any non-base-case number is of the form S(k) for some k. By the inductive hypothesis, p(k) holds. Use this fact to prove p(S(k)).