snick  snack CPSC 121: Models of Computation 2010 Winter Term 2 Revisiting Induction Steve Wolfman, based on work by Patrice Belleville and others 1

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 2

Learning Goals: Pre-Class By the start of class, you should be able to: –Given a theorem to prove stated in terms of its induction variable (i.e., usually, in terms of n), write out the skeleton of an inductive proof including: the basis step(s) that need to be proven, the inductive hypothesis, and the inductive step that needs to be proven. 3

Learning Goals: In-Class By the end of this unit, you should be able to: –Formally prove properties of the non-negative integers (or a subset like integers larger than 3) that have appropriate self-referential structure— including both equalities and inequalities—using either weak or strong induction as needed. –Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid. 4

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 5

Induction: a Domino Effect A proof by induction is like toppling a chain of dominoes. Let’s say I tell you: –the first domino in the chain toppled –for every domino in the chain, if the domino before it toppled, then the domino itself toppled Did all the dominoes in the chain topple? 6

Formal (Weak) Induction A formal induction proof follows the same pattern. To prove some property P(. ) applies to all positive integers n, we prove: The first domino topples. If one domino topples… Then the next one topples. P(0) is true. [For arbitrary k] If P(k) is true... Then P(k+1) is also true. Typically, we prove the second step by antecedent assumption/direct proof. 7

Form of an Induction Proof Theorem: some property that depends on n Basis step: show that a small case works Inductive Hypothesis: Assume your property holds for n=k where k ≥ your basis step. Inductive Step: Under this assumption, you need to prove that your property holds for n=k+1. 8

Side Note: Practical Induction That’s how you formally write out the proof, but how do you figure out the proof? Start at the inductive step! 9

Side Note: Practical Induction Look at a “big” problem (of size n). Figure out how to break it down into smaller pieces. Assume those smaller pieces work. That will end up as your inductive hypothesis. Figure out which problems cannot be broken down (usually small ones!). Those will end up as your basis step(s). 10

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 11

Worked Problem: How Many Introductions? Problem: n people would like to introduce themselves to each other. How many introductions does it take? For 2 people? For 3 people? For 4 people? For 5 people? … For n people? Sound familiar? Let’s prove it. 12

Worked Problem: How Many Introductions? Induction can feel very abstract. Let’s do the first few steps concretely… For 1 person? Given the number for 1, for 2 ? Given the number for 2, for 3 ? 13

Worked Problem: How Many Introductions? And now the basis step plus the abstract connection between one step and the next… For 1 person? 0 introductions. Given the number for k (call it x ), for k+1 ? a. x + 1 b. x * k c. x + 2k d. x + 2(k + 1) e. None of these 14 My turn!

Formally… Definition: When two people greet each other, we count that as two introductions. To prove: It takes n(n-1) introductions for every pair of people in a group of n people to greet each other. 15

Proof Basis step: A group of size 1 takes 0 intros. (Because there are no pairs.) Note: we talked about induction to prove properties for the non-negative integers, but it can be more general. 16

Proof (Continued) Inductive step, to prove: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. Assume size k takes k(k-1) intros (the Inductive Hypothesis)... 17

Proof (Continued) Inductive step, continued: Imagine a group of size k that has completed introductions. That took k(k-1) intros by the IH. Add one more to the group, and that person must greet all k other people, for a total of k(k-1) + 2k introductions. k(k-1) + 2k = k[(k-1) + 2] = k(k+1) = (k+1)((k+1) – 1) QED: if k takes k(k-1), (k+1) takes (k+1)((k+1)-1). 18

Proof (Completed) Basis step: A group of size 1 takes 0 intros. (See proof above.) Inductive step: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. (See proof above.) QED. From these two we could (but don’t have to) crank out a proof for any n that a group of size n requires n(n-1) introductions. 19

Cranking the Machine Basis step: A group of size 1 takes 0 intros. (See proof above.) Inductive step: If a group of size k takes k(k-1) intros, then a group of size k+1 takes (k+1)((k+1)-1) intros. (See proof above.) Can you use these to prove the n=2 case? The n=5 case?a. Yes. The n= case?b. No. The n=  case?c. It depends. 20

One More Time: the Induction Pattern Establish the basis step –must lead to all the other cases –often P(1) or P(0), but not necessarily –often just one, but may be several Prove the inductive step –always adds a case based on “smaller” cases –weak induction: if P(k) then P(k+1) –“strong” induction: if P(0), P(1),..., P(k-2), and P(k-1), then P(k+1). –generally: assume whatever you need! QED, proven for all cases reachable by the inductive step from the basis step (usually all N) 21

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 22

Historical Problem: Sum of Odd Numbers Problem: What is the sum of the first n odd numbers? First, find the pattern. Then, prove it’s correct. The first 1 odd number? The first 2 odd numbers? The first 3 odd numbers? The first n odd numbers? Historical note: Francesco Maurolico made the first recorded use of induction in 1575 to prove this theorem! 23

Historical Problem: Sum of Odd Numbers Now, how do we prove it? We need... Property: For all positive integers n... ? Basis Step: Establish for n=? Inductive Hypothesis: Assume... ? Inductive Step: To prove... ? 24

Historical Problem: Sum of Odd Numbers Property: For all positive integers n, the sum of the first n odd natural numbers is n 2. (We have figure this out on our own... Induction doesn’t help!) Basis Step: Establish for n=? Inductive Hypothesis: Assume... ? Inductive Step: To prove... ? 25

Historical Problem: Sum of Odd Numbers Theorem: For all positive integers n, the sum of the first n odd natural numbers is n 2. Basis Step: Establish for n=1. The sum of the “first 1 odd natural numbers” is 1, which equals 1 2. Inductive Hypothesis: Assume... ? Inductive Step: To prove... ? 26

Historical Problem: Sum of Odd Numbers Theorem: For all positive integers n, the sum of the first n odd natural numbers is n 2. Basis Step for n = 1 Inductive Hypothesis: For some arbitrary k  1, assume the sum of the first k odd natural numbers is k 2 Inductive Step: To prove... ? 27

Historical Problem: Sum of Odd Numbers Theorem: As above. Basis Step for n = 1 Inductive Hypothesis... Inductive Step: Prove that the sum of the first k+1 odd natural numbers is (k+1) 2. The (k+1) th odd number is 2(k+1) - 1 = 2k + 1 (k+1) 2 = k 2 + 2k + 1 = k 2 + [(k+1) th odd number] By the IH, k 2 = sum of the first k odd numbers; so, (k+1) 2 = sum of the first k+1 odd numbers. QED 28

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 29

Problem: Proof Critique Theorem: All horses are the same colour. See handout. Problem: Critique the proof. 30

Problem: Proof Critique Theorem: All horses are the same colour. See handout. Proof critique: Is the proof valid? a.Yes, because each step follows irrefutably from the previous steps. b.Yes, because the premises are false. c.Yes, but not for the reasons listed here. d.No, because the conclusion is false. e.No, but not for the reasons listed here. 31

Problem: Proof Critique Theorem: All horses are the same colour. See handout. Proof critique: Where is the error in the proof? a.The basis step. b.The inductive hypothesis. c.The inductive step. d.None of these, but there must be an error. e.None of these, and there may be no error. 32

Problem: Proof Critique Theorem: All horses are the same colour. See handout. Proof critique: Can the proof be fixed? 33

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 34

Induction and Computer Science How important is induction? Is it just for proving things about N? Induction forms the basis of… –proofs of correctness and efficiency for many algorithms: e.g., How long does it take to detect duplicates in a list of n inputs? Does binary search work? –“recursive” algorithms: e.g., “merge sort” –“recursive data structures”: e.g., How big can a “tree” of height n be? 35

Problem: Detecting Duplicates Problem: How long does it take to detect duplicates in a list of n Inputs? Let’s assume the amount of “time” taken is proportional to the number of comparisons we make between numbers. We could compare each element to each other element and see if they’re the same. Sound familiar? We already know how long this takes! Note: this isn’t the fastest algorithm! 36

Problem: Binary Search Works Problem: Prove that binary search works. Binary search is when we search a sorted list by checking the middle element. If it’s what we’re looking for, we’re done. Otherwise, we “throw out” the part of the list we don’t need (e.g., the left half if the element we looked at was too small) and start over on what remains. 37

Form of a Strong Induction Proof Theorem: some property that depends on n Basis step: show that one or more small cases (as needed) work Inductive Hypothesis: Assume your property holds for any n = i where your basis step ≤ i your basis step. Inductive Step: Under this assumption, you need to prove that your property holds for n = k. Hint: there’s really no difference. Just assume what you need in order to break the problem into smaller pieces. 38

Side Note: “Strong Induction” We’ve used weak induction: basis step ( n = a ) plus “if it works for n then it works for n+1 ”. In strong induction, we use: basis step plus “if it works for all a  i < k then it works for k ”. Can we still build our individual proofs (for n = 1, n = 2, n = 3,...) from that? 39

Sorting by “Merging” Problem: sort a list of names. Algorithm: 1.If the list is of length 1, it’s sorted. 2.Otherwise: a)Divide the list in half (or as close as possible). b)Sort each half using this algorithm. c)Merge the sorted lists back together. 40

The Merge Step Problem: given two lists of names in sorted order, merge them into a single sorted list. Algorithm to merge lists a and b: 1.Let c be an empty list. 2.While a or b has elements remaining: a)If one of a and b is empty, remove the first element from the other list and put it at the end of c. b)Otherwise, compare the first elements of a and b, remove the smaller one, and put it at the end of c. 3.Return c. 41

Problem: Does Merging Work? Problem: Prove that merging works. Hint: the key to proving that a loop works is to use a “loop invariant”, some property that is true each time we start the loop. In this case, try focusing on what’s true of c on the n th time the loop starts. Your property should be true on the 0 th time the loop starts (when c is empty) and on the last time the loop starts (when c is, hopefully, the whole sorted list!). 42

Merging Invariant Which of these is not an invariant we might prove for merging? a.A is sorted b.A and B are the same length c.C contains i items (when we are about to execute the ith iteration of the loop) d.Items in C are smaller than items in A and B e.C is sorted 43

Merging Works: Basis Step When we are about to start the “0 th ” iteration of the loop... A is sorted? B is sorted? C is sorted? C contains i = 0 items? A+B contains n-i = n items? Items in C are smaller than items in A, B? 44

Merging Works: Inductive Hypothesis ASSUME: When we are about to start the k th iteration of the loop... A is sorted. B is sorted. C is sorted. C contains i = k items. A+B contains n-i = n-k items. Items in C are smaller than items in A, B. 45

Merging Works: Inductive Step PROVE: When we are about to start the (k+1) th iteration of the loop... A is sorted. B is sorted. C is sorted. C contains i = k+1 items. A+B contains n-i = n-k-1 items. Items in C are smaller than items in A, B. 46

Merging Works: Inductive Step Which one is easiest to prove? a.A is sorted. b.C is sorted. c.C contains i = k+1 items. d.A+B contains n-i = n-k-1 items. e.Items in C are smaller than items in A, B. 47

Problem: Merge Sort Works Problem: Now that we know the merge step works, prove that merge sort works. Algorithm: 1.If the list is of length 1, it’s sorted. 2.Otherwise: a)Divide the list in half (or as close as possible). b)Sort each half using this algorithm. c)Merge the sorted lists back together. 48

Problem: Merge Sort Works Basis Step: The algorithm does nothing to a list of length 1. Lists of length 1 are trivially sorted. Therefore, the algorithm is correct for n = 1. 49

Problem: Merge Sort Works Inductive Hypothesis: Assume the algorithm correctly sorts any list of length 1  i 1. Inductive Step: Prove that the algorithm correctly sorts a list of length k. 50

Problem: Merge Sort Works Inductive Hypothesis: Assume the algorithm correctly sorts any list of length 1  i 1. Inductive Step: On a list of length k, the algorithm divides the list into two parts, one of length  k/2 , the other of length  k/2 . k is at least 2; so, each half has at least one element (and is smaller than the original). So, the IH applies. 51

Problem: Merge Sort Works Inductive Step: Continued... By the IH, the algorithm correctly sorts the two smaller lists. It then merges them. We’ve already proved that the merge step merges these into a single sorted list. Therefore, the algorithm correctly sorts the list of length k. QED 52

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 53

Problem: Prove that 2 n < n! Problem: prove that 2 n < n! ? Note: is 2 n < n! ? 54

“Rules” for Inequalities To prove an inequality, do what you need in scratch work, then rewrite formally: Start from one side. Work step-by-step to the other. Never move “opposite” to your inequality (so, to prove “<“, never make the quantity smaller). Strict inequalities: have at least one strict inequality step. 55

Outline Prereqs and Learning Goals Induction as a Formal Argument Form Problems and Discussion –Introductions –Odd Numbers –Horse Colours –CS Induction: Duplicate Detection, Binary Search, MergeSort –More examples Next Lecture Notes 56

Learning Goals: In-Class By the end of this unit, you should be able to: –Formally prove properties of the non-negative integers (or a subset like integers larger than 3) that have appropriate self-referential structure— including both equalities and inequalities—using either weak or strong induction as needed. –Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid. 57

snick  snack Extra Slides 58

Problem: Proof Critique Theorem: All integers ≥ 2 are even. Problem: Critique the proof. Is it valid? If not, why not? Can it be fixed, and how? 59

Problem: Sum of a Geometric Series Problem: What is the sum of the first n terms of the form a i, if a is a real number between 0 and 1 ? (Note: we mean terms 0 through n.) Note: we’re looking for Can we use induction over the real numbers between 0 and 1? 60