CSC317 1 So far so good, but can we do better? Yes, cheaper by halves... orkbook/cheaperbyhalf.html Let’s try it out …

CSC317 2 Toward merge sort

CSC317 3 What is it going to cost us? Divide Left and Right: constant C 1 Sort Left with Insertion Sort: SortRight:withInsertionSort Merge: C 3 n Total: (Have we gained from Insertion sort ( Cn 2 )?

CSC317 4 Another example: find min Input: array A Output: minimum value in array A Cost? C 1 n

CSC317 5 Is splitting the array in half going to help us? Let’s see: Divide Left and Right: constant C 1 Find min Left: Find min Right: Merge: C 3 n Total: Have we gained from find min on full array ( Cn )?

CSC317 6 Back to merge sort: If we can split once, can we do it more often?

CSC317 7 Merge sort: pseudo code (step-by-step) Merge-Sort If array larger than size 1 Divide array into Left and Right arrays // divide Merge-Sort(Left array) // conquer left; recursive call Merge-Sort(Right array) // conquer right; recursive call Merge sorted Left and Right arrays // combine

CSC317 8 Merge sort: pseudo code (more formal) length of array A // divide (split in half) // conquer left // conquer right // combine At what step do we have most work? Merging step, otherwise we just split arrays

CSC317 9 Merge sort: run time analysis Total work in each step: Divide: constant Combine: Cn Conquer: recursively solve two subproblems, each size n/2 We’ll write out the recursion as combination recursion grows like n log 2 n Compared to insertion sort is this good or bad? And how did we end up here?

CSC What is this? A recursion tree where combination recursion is Cn

CSC What is this? A recursion tree where combination recursion is Cn

CSC Cn Next step in the recursion: Cn/2

CSC Cn Let’s keep on going (I mean splitting)! Cn/2 Cn/4 C C C CCCC Each row adds to how much work? C

CSC Cn Cn/2 Cn/4 C C C CCCC C Cn Cost per level stays the same!

CSC Cn Cn/2 Cn/4 C C C CCCC C Level 1: Level 2: Level 0: Level k:

CSC We know that n = 1 and We need to find level k (height of tree) number of levels, each level needs work Cn Total work: Cn lg 2 (n) CONCLUSION: grows as n lg 2 (n) Insert sort: n 2 Is mergesort always faster? Any disadvantages?

CSC Correctness and loop invariants How do we know that an algorithm is correct (i.e. always gives the right answer? We use loop invariants (what does that mean?). Invariant = something that does not change Loop invariant = a property about the algorithm that does not change at every iteration before the loop Usually the property we would like to prove is correct about the algorithm! Intuitive, but we would like to state mathematically

CSC Loop invariant example: insertion sort Question: What invariant property makes this algorithm correct?

CSC Loop invariant example: insertion sort Question: What invariant property makes this algorithm correct? Answer: Before each iteration of the for loop, the elements thus far are sorted. Next question: Can we state that mathematically? KEY

CSC Loop invariant example: insertion sort KEY Insertion sort loop invariant: at the start of each iteration of the for loop, A[1,...,j-1] consists of elements originally in A[1,...,j-1] but in sorted order.

CSC Can we prove this? 3 steps: Initialization: Algorithm is true prior to first iteration of the loop (base case). Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). Termination: When the loop terminates, the invariant gives a useful property that shows the algorithm is correct.

CSC Initialization: Algorithm is true prior to first iteration of the loop (base case). When j=2, A[1] is just one element, which is the original element in A[1], and must be already sorted 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position Insert sort pseudocode j=2

CSC Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY true for j-1 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position Might not be true in loop. But make sure here that it remains sorted, by pairwise swaps.

CSC Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY So will remain true for j 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position We make sure here that it remains sorted, by pairwise swaps

CSC Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY If A[1,...,j-1] sorted before iteration of loop, then for key=A[j], we pairwise swap it into correct position; so now A[1,...,j] is also sorted. Also, A[1,...,j-1] includes elements originally in A[1,...,j-1]. Then A[1,...,j] includes those elements and the element A[j], so must include elements originally in A[1,...,j]

CSC Termination: When the loop terminates, the invariant gives a useful property that shows the algorithm is correct. When the loop terminates (i.e. j=n+1) A[1,...,j] must be in sorted order, which is A[1,…,n] or the entire array. true for j= n for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position

CSC Loop invariants example: find min Animation example (Burt Rosenberg) Input: array A Output: minimum value in array A Question: What invariant would make this algorithm correct?

CSC Input: array A Output: minimum value in array A Question: What invariant would make this algorithm correct? Answer: At each point of the algorithm, the current min is known. More formally …

CSC Input: array A Output: minimum value in array A Loop invariant: At each iteration of the for loop, min is the smallest Element in A[1,…,i-1].

CSC Loop invariant of merge: Loop invariant: At the start of each iteration of the for loop, A[p,...,k-1] contains the k-p smallest elements, in sorted order. Also, L[i] and R[j] are the smallest elements of their arrays not yet copied back into A. k J k i i j