1. CSC317 1 So far so good, but can we do better? Yes, cheaper by halves... http://www.cs.miami.edu/~burt/learning/Csc517.101/w orkbook/cheaperbyhalf.html Let’s try it out …

2. CSC317 2 Toward merge sort

3. CSC317 3 What is it going to cost us? Divide Left and Right: constant C 1 Sort Left with Insertion Sort: SortRight:withInsertionSort Merge: C 3 n Total: (Have we gained from Insertion sort ( Cn 2 )?

4. CSC317 4 Another example: find min Input: array A Output: minimum value in array A Cost? C 1 n

5. CSC317 5 Is splitting the array in half going to help us? Let’s see: Divide Left and Right: constant C 1 Find min Left: Find min Right: Merge: C 3 n Total: Have we gained from find min on full array ( Cn )?

6. CSC317 6 Back to merge sort: If we can split once, can we do it more often?

7. CSC317 7 Merge sort: pseudo code (step-by-step) Merge-Sort If array larger than size 1 Divide array into Left and Right arrays // divide Merge-Sort(Left array) // conquer left; recursive call Merge-Sort(Right array) // conquer right; recursive call Merge sorted Left and Right arrays // combine

8. CSC317 8 Merge sort: pseudo code (more formal) length of array A // divide (split in half) // conquer left // conquer right // combine At what step do we have most work? Merging step, otherwise we just split arrays

9. CSC317 9 Merge sort: run time analysis Total work in each step: Divide: constant Combine: Cn Conquer: recursively solve two subproblems, each size n/2 We’ll write out the recursion as combination recursion grows like n log 2 n Compared to insertion sort is this good or bad? And how did we end up here?

10. CSC317 10 What is this? A recursion tree where combination recursion is Cn

11. CSC317 11 What is this? A recursion tree where combination recursion is Cn

12. CSC317 12 Cn Next step in the recursion: Cn/2

13. CSC317 13 Cn Let’s keep on going (I mean splitting)! Cn/2 Cn/4 C C C CCCC Each row adds to how much work? C

14. CSC317 14 Cn Cn/2 Cn/4 C C C CCCC C Cn Cost per level stays the same!

15. CSC317 15 Cn Cn/2 Cn/4 C C C CCCC C Level 1: Level 2: Level 0: Level k:

16. CSC317 16 We know that n = 1 and We need to find level k (height of tree) number of levels, each level needs work Cn Total work: Cn lg 2 (n) CONCLUSION: grows as n lg 2 (n) Insert sort: n 2 Is mergesort always faster? Any disadvantages?

17. CSC317 17 Correctness and loop invariants How do we know that an algorithm is correct (i.e. always gives the right answer? We use loop invariants (what does that mean?). Invariant = something that does not change Loop invariant = a property about the algorithm that does not change at every iteration before the loop Usually the property we would like to prove is correct about the algorithm! Intuitive, but we would like to state mathematically

18. CSC317 18 Loop invariant example: insertion sort Question: What invariant property makes this algorithm correct?

19. CSC317 19 Loop invariant example: insertion sort Question: What invariant property makes this algorithm correct? Answer: Before each iteration of the for loop, the elements thus far are sorted. Next question: Can we state that mathematically? KEY

20. CSC317 20 Loop invariant example: insertion sort KEY Insertion sort loop invariant: at the start of each iteration of the for loop, A[1,...,j-1] consists of elements originally in A[1,...,j-1] but in sorted order.

21. CSC317 21 Can we prove this? 3 steps: Initialization: Algorithm is true prior to first iteration of the loop (base case). Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). Termination: When the loop terminates, the invariant gives a useful property that shows the algorithm is correct.

22. CSC317 22 Initialization: Algorithm is true prior to first iteration of the loop (base case). When j=2, A[1] is just one element, which is the original element in A[1], and must be already sorted 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position Insert sort pseudocode j=2

23. CSC317 23 Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY true for j-1 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position Might not be true in loop. But make sure here that it remains sorted, by pairwise swaps.

24. CSC317 24 Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY So will remain true for j 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position We make sure here that it remains sorted, by pairwise swaps

25. CSC317 25 Maintenance: If it is true before an iteration of the loop it remains true before the next iteration (like an induction step). KEY If A[1,...,j-1] sorted before iteration of loop, then for key=A[j], we pairwise swap it into correct position; so now A[1,...,j] is also sorted. Also, A[1,...,j-1] includes elements originally in A[1,...,j-1]. Then A[1,...,j] includes those elements and the element A[j], so must include elements originally in A[1,...,j]

26. CSC317 26 Termination: When the loop terminates, the invariant gives a useful property that shows the algorithm is correct. When the loop terminates (i.e. j=n+1) A[1,...,j] must be in sorted order, which is A[1,…,n] or the entire array. true for j= n + 1 1. for j = 2 to n 2. key = A[ j ] 3. Insert key into sorted array A[1,...,j-1] by comparing and swapping into correct position

27. CSC317 27 Loop invariants example: find min Animation example (Burt Rosenberg) http://www.cs.miami.edu/~burt/learning/Csc517.101/workbook/findmin.html Input: array A Output: minimum value in array A Question: What invariant would make this algorithm correct?

28. CSC317 28 Input: array A Output: minimum value in array A Question: What invariant would make this algorithm correct? Answer: At each point of the algorithm, the current min is known. More formally …

29. CSC317 29 Input: array A Output: minimum value in array A Loop invariant: At each iteration of the for loop, min is the smallest Element in A[1,…,i-1].

30. CSC317 30 Loop invariant of merge: Loop invariant: At the start of each iteration of the for loop, A[p,...,k-1] contains the k-p smallest elements, in sorted order. Also, L[i] and R[j] are the smallest elements of their arrays not yet copied back into A. k J k i i j