CPSC 121: Models of Computation 2008/9 Winter Term 2 Revisiting Induction Steve Wolfman, based on work by Patrice Belleville and others

Lecture Prerequisites Read Section 4.2-4.4. Solve (or at least “set up”, by writing the base case(s) that need to be proved, the induction hypothesis, and the inductive step that needs to be proved) problems like Exercise Set 4.2 #1-2, 5-17, and 31; 4.3 #1, 8-27, 29, and 30-31, and 4.4 #1-7, 9-10, and 24. Solve problems like Exercise Set 4.2 #3-4 and 32, 4.3 #6-7 and 30-31, and 4.4 #17. Complete the open-book, untimed quiz on WebCT that’s due before the next class.

Learning Goals: Pre-Class By the start of class, you should be able to: Given a theorem to prove stated in terms of its induction variable (i.e., usually, in terms of n), write out the skeleton of an inductive proof including: the base case(s) that need to be proven, the induction hypothesis, and the inductive step that needs to be proven. In other words, take what we did in this lecture and use the textbook readings to formalize your understanding. We will have much more practice on inductive proofs. Discuss point of learning goals.

Learning Goals: In-Class By the end of this unit, you should be able to: Formally prove properties of the non-negative integers (or a subset like integers larger than 3) that have appropriate self-referential structure—including both equalities and inequalities—using either weak or strong induction as needed. Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid. Discuss point of learning goals.

Outline Induction: Metaphorically and Formally Induction: Worked, Open, and Critique Induction in CS: Finding Duplicates, Binary Search, and MergeSort More Practice: Inequalities and Critique

Induction: a Domino Effect A proof by induction is like toppling a chain of dominoes. Let’s say I tell you: the first domino in the chain toppled for every domino in the chain, if the domino before it toppled, then the domino itself toppled Did all the dominoes in the chain topple?

Formal (Weak) Induction A formal induction proof follows the same pattern. To prove some property P(.) applies to all positive integers n, we prove: The first domino topples. If one domino topples… Then the next one topples. P(0) is true. If P(k) is true (for arbitrary k)… Then P(k+1) is also true. Typically, we prove the second step by antecedent assumption/direct proof.

Form of an Induction Proof Theorem: some property that depends on n Base case: show that a small case works Induction Hypothesis: Assume your property holds for an arbitrary integer n where n ≥ your base case. Inductive Step: Under this assumption, you need to prove that your property holds for n+1.

Outline Induction: Metaphorically and Formally Induction: Worked, Open, and Critique Induction in CS: Finding Duplicates, Binary Search, and MergeSort More Practice: Inequalities and Critique

Worked Problem: How Many Introductions? Problem: n people would like to introduce themselves to each other. How many introductions does it take? For 2 people? For 3 people? For 4 people? For 5 people? … For n people? Sound familiar? Let’s prove it.

Worked Problem: How Many Introductions? Induction can feel very abstract. Let’s do the first few steps concretely… For 1 person? Given the number for 1, for 2? Given the number for 2, for 3? Talk about use of mathematical techniques and notation to represent the problem and its solution and to prove the correctness of the solution.

Worked Problem: How Many Introductions? And now the base case plus the abstract connection between one step and the next… For 1 person? Given the number for k, for k+1? Talk about use of mathematical techniques and notation to represent the problem and its solution and to prove the correctness of the solution.

Formally… Definition: When two people greet each other, we count that as two introductions. To prove: It takes n(n-1) introductions for every pair of people in a group of n people to greet each other.

Proof Base case: A group of size 1 takes 0 intros. (Because there are no pairs.) Note: we talked about induction to prove properties for the non-negative integers, but it can be more general.

Proof (Continued) “Inductive” step, to prove: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. Assume size k takes k(k-1) intros (the Induction Hypothesis) Imagine a group of size k that has completed introductions. That took k(k-1) intros by the IH. Add one more to the group, and that person must greet all k other people, for a total of k(k-1) + 2k introductions. k(k-1) + 2k = k[(k-1) + 2] = k(k+1) = (k+1)((k+1) – 1) QED: if k takes k(k-1), (k+1) takes (k+1)((k+1)-1). Split into several slides.

Proof (Completed) Base case: A group of size 1 takes 0 intros. (See proof above.) “Inductive” step: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. (See proof above.) QED. So, there is a proof with a finite number of steps for every n that a group of size n requires n(n-1) introductions.

Cranking the Machine Base case: A group of size 1 takes 0 intros. (See proof above.) “Inductive” step: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. (See proof above.) Can you use these to prove the n=2 case? The n=5 case? The n=8675309 case? The n= case?

One More Time: the Induction Pattern Establish the base case must lead to all the other cases often P(1) or P(0), but not necessarily often just one, but may be several Prove the inductive step always adds a case based on “smaller” cases weak induction: if P(k) then P(k+1) “strong” induction: assume P(.) for any values you need smaller than k+1, prove P(k+1). QED, proven for all cases reachable by the inductive step from the base case (usually all N) Metaphors: - zombie movie (first zombie bites others, etc.) - ???

Historical Problem: Sum of Odd Numbers Problem: What is the sum of the first k odd numbers? First, find the pattern. Then, prove it’s correct. The first 1 odd number? The first 2 odd numbers? The first 3 odd numbers? The first k odd numbers? Historical note: Francesco Maurolico made the first recorded use of induction in 1575 to prove this theorem!

Problem: Proof Critique Theorem: All horses are the same colour. See handout. Problem: Critique the proof. Is it valid? If not, why not? Can it be fixed, and how?

Outline Induction: Metaphorically and Formally Induction: Worked, Open, and Critique Induction in CS: Finding Duplicates, Binary Search, and MergeSort More Practice: Inequalities and Critique

Induction and Computer Science How important is induction? Is it just for proving things about N? Induction forms the basis of… proofs of correctness and efficiency for many algorithms: e.g., How long does it take to detect duplicates in a list of n inputs? Does binary search work? “recursive” algorithms: e.g., “merge sort” “recursive data structures”: e.g., How big can a “tree” of height k be?

Problem: Detecting Duplicates Problem: How long does it take to detect duplicates in a list of n Inputs? Let’s assume the amount of “time” taken is proportional to the number of comparisons we make between numbers. Sound familiar? Let’s pick an algorithm we already know… Have each number “introduce” itself to each other number. If they’re the same, we have a duplicate. Note: this isn’t the fastest algorithm!

Problem: Binary Search Works Problem: Prove that binary search works. Binary search is when we search a sorted list by checking the middle element. If it’s what we’re looking for, we’re done. Otherwise, we “throw out” the part of the list we don’t need (e.g., the left half if the element we looked at was too small) and start over on what remains.

Form of a Strong Induction Proof Theorem: some property that depends on n Base case: show that one or more small cases (as needed) work Induction Hypothesis: Assume your property holds for any integer k where your base case ≤ k < n and n is an integer > your base case. Inductive Step: Under this assumption, you need to prove that your property holds for n. In general, assume what you need for your inductive step to work as long as you break the problem into smaller pieces.

Side Note: “Strong Induction” We’ve used weak induction: base case plus “if it works for n then it works for n+1”. In strong induction, we use: base case plus “if it works for all k < n then it works for n”. Can we still build our individual proofs (for n = 1, n = 2, n = 3, ...) from that?

Sorting by “Merging” Problem: sort a list of names. Algorithm: If the list is of length 1, it’s sorted. Otherwise: Divide the list in half (or as close as possible). Sort each half using this algorithm. Merge the sorted lists back together.

Problem: Merge Sort Works Problem: Prove that merge sort works. (For now, assume the “merge” step works.)

The Merge Step Problem: given two lists of names in sorted order, merge them into a single sorted list. Algorithm to merge lists a and b: Let c be an empty list. While a or b has elements remaining: If one of a and b is empty, remove the first element from the other list and put it at the end of c. Otherwise, compare the first elements of a and b, remove the smaller one, and put it at the end of c. Return c.

Problem: Does Merging Work? Problem: Prove that merging works. Hint: the key to proving that a loop works is to use a “loop invariant”, some property that is true each time we start the loop. In this case, try focusing on what’s true of c on the nth time the loop starts. Your property should be true on the 0th time the loop starts (when c is empty) and on the last time the loop starts (when c is, hopefully, the whole sorted list!).

Outline Induction: Metaphorically and Formally Induction: Worked, Open, and Critique Induction in CS: Finding Duplicates, Binary Search, and MergeSort More Practice: Inequalities, Choosing Induction Variables, and Critique

Problem: Prove that 2n < n! Note: is 2n < n!?

“Rules” for Inequalities To prove an inequality, do what you need in scratch work, then rewrite formally: Start from one side. Work step-by-step to the other. Never move “opposite” to your inequality (so, to prove “<“, never make the quantity smaller). Strict inequalities: have at least one strict inequality step.

Problem: Sum of a Geometric Series Problem: What is the sum of the first n terms of the form ai, if a is a real number between 0 and 1? (Note: we mean terms 0 through n.) Note: we’re looking for It turns out the question about the reals is an incredibly deep one, tied up with the Axiom of Choice. If the Axiom of Choice holds, then the reals are well-ordered (and so we could use induction on them??). Else, they’re not. Unfortunately, even if the Axiom is true (which it provably isn’t NECESSARILY in our usual formalization of set theory), it doesn’t tell us what the well-ordering actually IS. Short version: no. Note: (a^(n+1) – 1)/(a – 1) Can we use induction over the real numbers between 0 and 1?

Problem: Proof Critique Theorem: All integers ≥ 2 are even. Problem: Critique the proof. Is it valid? If not, why not? Can it be fixed, and how?

Learning Goals: In-Class By the end of this unit, you should be able to: Formally prove properties of the non-negative integers (or a subset like integers larger than 3) that have appropriate self-referential structure—including both equalities and inequalities—using either weak or strong induction as needed. Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid. Discuss point of learning goals.