Sorting Chapter 6 Chapter 6 –Insertion Sort 6.1 –Quicksort 6.2 Chapter 5 Chapter 5 –Mergesort 5.2 –Stable Sorts Divide & Conquer

Three Algorithms to know Insertion Sort (not divide-and-conquer) Insertion Sort (not divide-and-conquer) –O(n 2 ) worst-case, best-case, average-case –very efficient on partially sorted lists. Mergesort (divide-and-conquer) Mergesort (divide-and-conquer) –O(n log n) worst-case, best-case, average-case –stable but not the most efficient Quicksort (divide-and-conquer) Quicksort (divide-and-conquer) –O(n 2 ) worst-case but O(n log n) average case. –Improvements make it the fastest practical sorting algorithm

1. Insertion Sort is(a[], n) { // Assume first item a[o] is sorted for i = 1 to n-1 { // Sort a[1] to a[n-1] v = a[i];// v is item we want to insert j = i –1;// j will iterate from i down to 0 // Keep looping until you insert v while (j >= 0 && v = a[j] a[j+1] = a[j]; // Shift to make room j--;} a[j+1] = v;// Insert v into correct position }}

1. Insertion Sort 10 items Worst Case 10*9/2 = 45 comparisons n*(n-1)/2 = O(n 2 )

1. Insertion Sort 10 items Best Case 9 comparisons O(n)

1. Insertion Sort 10 items Average Case

2. Mergesort mergsort(a[], i, j) { if (i == j) return;// If sub-list is size 1 m = (i+j)/2;// Compute mid-point mergesort(a, i, m);// Mergesort first half mergesort(a, m+1, j); // Mergesort second half merge(a, i, m, j); // Merge the two halves // Merging is O(n), where n = a + b, where a and b are the size of the two sublists. }

2. Mergesort mergsort(a[], i, j) { if (i == j) return;// Base case m = (i+j)/2;// O(1) mergesort(a, i, m);// Recursive call mergesort(a, m+1, j); // Recursive call merge(a, i, m, j); // O(n) }

Recall the Big Hammer: a = # recursive calls b = 2 (if you cut the input size in half) n k  is the running time of the actual function (irrespective of recursion) 1. a < b k T(n) = Θ(n k ) 2. a = b k T(n) = Θ(n k log n ) 3. a > b k T(n) = Θ( n log b a )

2. The Merge Function ?????????

??????10??

??????1012?

?????1012?

????1012?

???1012?

??1012?

?1012?

?

2. Merge Function While the merge function does “minimal” comparisons, it must do a lot of “moves” i.e., swaps, copies, etc. While the merge function does “minimal” comparisons, it must do a lot of “moves” i.e., swaps, copies, etc. In fact, it is nearly impossible to do the merge efficiently without using extra memory. In fact, it is nearly impossible to do the merge efficiently without using extra memory. For each merge, all the items must be moved at least twice. For each merge, all the items must be moved at least twice. Moved once in the temp array and then back again. Moved once in the temp array and then back again.

2. Merge Function Notice the left and right sub-lists are sorted. Notice the left and right sub-lists are sorted. They must be sorted otherwise the merge function won’t work! period They must be sorted otherwise the merge function won’t work! period

2. Merge Function The first comparison does NOT require any swaps The first comparison does NOT require any swaps

2. Merge Function The second comparison does The second comparison does 10 is less than 30, so we should swap 10 is less than 30, so we should swap

2. Merge Function But, how do we continue the merge? But, how do we continue the merge? Which pointer should be moved? Which pointer should be moved?

2. Merge Function The values 1 and 10 or sorted, so moving the blue pointer makes sense, right? But where should we move it? The values 1 and 10 or sorted, so moving the blue pointer makes sense, right? But where should we move it? ??

2. Merge Function We can’t move it to the 40 because then we’ll compare 40 with 30 next, and we’ll miss 15, which is smaller. We can’t move it to the 40 because then we’ll compare 40 with 30 next, and we’ll miss 15, which is smaller ??

2. Merge Function Now, we are in big trouble: Now, we are in big trouble: 1. How exactly will the 15 get into the correct position? 2. What happened to our 2 sorted sub- lists? ?

2. Merge Function Catch-22 The only way to efficiently merge sorted sub-list is to use an extra temp array to do the swapping. The only way to efficiently merge sorted sub-list is to use an extra temp array to do the swapping. However, if you use linked list, then you can efficiently merge without extra memory However, if you use linked list, then you can efficiently merge without extra memory Double However, with link lists you can’ t easily jump to the mid-point of each sub- list, so extra iteration is required. Double However, with link lists you can’ t easily jump to the mid-point of each sub- list, so extra iteration is required.

2. Mergesort Conclusion Lesson: Minimizing comparisons is good Minimizing comparisons is good But using extra memory and But using extra memory and –having to move so much data around… …makes Mergesort one of the most inefficient O(n log n) sorting algorithms, no matter how you implement it. …makes Mergesort one of the most inefficient O(n log n) sorting algorithms, no matter how you implement it.

Quicksort quicksort (a[], i, j) { if (i < j) { p = partition(a, i, j); quicksort(a, i, p-1); quicksort(a, p+1, j); }} partition(a[],i,j) { v = a[i]; h = i; for k = i+1 to j if (a[k] < v) { h++;swap(a[h],a[k]);} swap (a[i],a[h]); return h; }

Quicksort 31 items Best Case O(n log n) log n levels O(n) work for each level

Quicksort 31 items Worst Case O(n 2 ) n levels O(n) work for each level...

Quicksort 31 items Average Case O(n log n) log n levels O(n) work for each level Proof on page

Quicksort vs. Mergesort Mergesort -close to minimum number of comparisons Mergesort -close to minimum number of comparisons –But every comparison requires moving two values a[i]  c[?]  a[?]. –Mergesort always moves 2n log n values. Quicksort - does more comparisons, but less moves or swaps Quicksort - does more comparisons, but less moves or swaps –Also, once the pivot is placed, it is never moved, i.e., its in the correct position.

Important Concept General Sorting General Sorting –The only thing you are allowed to do is compare two items at a time. Special Sorting Special Sorting –It is possible to compare one item with every other item in one operation. –How so?

Decision Tree for Sorting a<=b abc b <= c a <= c b <= c abc bac acbcabbcacba

Decision Tree for Sorting a<=b abc b <= c a <= c d 12 permutations 6 permutations 3 permutations 2 permutations?

Lower-bound on General Sorting h >= log(n!) h >= log(n!) h is the minimum number of comparisons h is the minimum number of comparisons log(n!) =  (n log n) (well known 2.3.8) log(n!) =  (n log n) (well known 2.3.8) Thus, log(n!) =  (n log n) and Thus, log(n!) =  (n log n) and h =  (n log n) h =  (n log n)