Sorting Dr. Bernard Chen Ph.D. University of Central Arkansas Fall 2010

Insertion Sort I The list is assumed to be broken into a sorted portion and an unsorted portion Keys will be inserted from the unsorted portion into the sorted portion. Sorted Unsorted

Insertion Sort II For each new key, search backward through sorted keys Move keys until proper position is found Place key in proper position Moved

Insertion Sort: Code template void insertionSort( vector & a ) { for( int p = 1; p < a.size( ); p++ ) { Comparable tmp = a[ p ]; int j; for( j = p; j > 0 && tmp < a[ j - 1 ]; j-- ) a[ j ] = a[ j - 1 ]; a[ j ] = tmp; } Fixed n-1 iterations Worst case i-1 comparisons Move current key to right Insert the new key to its proper position Searching for the proper position for the new key Moved

Insertion Sort: Analysis Worst Case: Keys are in reverse order Do i-1 comparisons for each new key, where i runs from 2 to n. Total Comparisons: … + n-1 Comparison

Optimality Analysis I To discover an optimal algorithm we need to find an upper and lower asymptotic bound for a problem. An algorithm gives us an upper bound. The worst case for sorting cannot exceed  (n 2 ) because we have Insertion Sort that runs that fast. Lower bounds require mathematical arguments.

Other Assumptions The only operation used for sorting the list is swapping two keys. Only adjacent keys can be swapped. This is true for Insertion Sort and Bubble Sort.

Shell Sort With insertion sort, each time we insert an element, other elements get nudged one step closer to where they ought to be What if we could move elements a much longer distance each time? We could move each element: A long distance A somewhat shorter distance A shorter distance still This approach is what makes shellsort so much faster than insertion sort

Sorting nonconsecutive subarrays  Consider just the red locations  Suppose we do an insertion sort on just these numbers, as if they were the only ones in the array?  Now consider just the yellow locations  We do an insertion sort on just these numbers  Now do the same for each additional group of numbers  The resultant array is sorted within groups, but not overall Here is an array to be sorted (numbers aren’t important)

Doing the 1-sort In the previous slide, we compared numbers that were spaced every 5 locations This is a 5-sort Ordinary insertion sort is just like this, only the numbers are spaced 1 apart We can think of this as a 1-sort Suppose, after doing the 5-sort, we do a 1-sort? In general, we would expect that each insertion would involve moving fewer numbers out of the way The array would end up completely sorted

Example of shell sort original sort sort sort

Diminishing gaps For a large array, we don’t want to do a 5- sort; we want to do an N-sort, where N depends on the size of the array N is called the gap size, or interval size

Diminishing gaps We may want to do several stages, reducing the gap size each time For example, on a 1000-element array, we may want to do a 364-sort, then a 121- sort, then a 40-sort, then a 13-sort, then a 4-sort, then a 1-sort Why these numbers?

Increment sequence No one knows the optimal sequence of diminishing gaps This sequence is attributed to Donald E. Knuth: Start with h = 1 Repeatedly compute h = 3*h + 1 1, 4, 13, 40, 121, 364, 1093 This sequence seems to work very well

Increment sequence Another increment sequence mentioned in the textbook is based on the following formula: start with h = the half of the container’s size h i = floor (h i-1 / 2.2) It turns out that just cutting the array size in half each time does not work out as well

Analysis What is the real running time of shellsort? Nobody knows! Experiments suggest something like O(n 3/2 ) or O(n 7/6 ) Analysis isn’t always easy!

Merge Sort If List has only one Element, do nothing Otherwise, Split List in Half Recursively Sort Both Lists Merge Sorted Lists Mergesort(A, l, r) if l < r then q = floor((l+r)/2) mergesort(A, l, q) mergesort(A, q+1, r) merge(A, l, q, r)

auxiliary array smallest AGLORHIMST Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. A

auxiliary array smallest AGLORHIMST A G Merging Merge. Merge. Keep track of smallest element in each sorted half. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Insert smallest of two elements into auxiliary array. Repeat until done. Repeat until done.

auxiliary array smallest AGLORHIMST AG Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. H

auxiliary array smallest AGLORHIMST AGH Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. I

auxiliary array smallest AGLORHIMST AGHI Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. L

auxiliary array smallest AGLORHIMST AGHIL Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. M

auxiliary array smallest AGLORHIMST AGHILM Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. O

auxiliary array smallest AGLORHIMST AGHILMO Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. R

auxiliary array first half exhausted smallest AGLORHIMST AGHILMOR Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. S

auxiliary array first half exhausted smallest AGLORHIMST AGHILMORS Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. T

auxiliary array first half exhausted second half exhausted AGLORHIMST AGHILMORST Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done.

Merging numerical example

30 Binary Merge Sort Given a single file Split into two files

31 Binary Merge Sort Merge first one-element "subfile" of F1 with first one-element subfile of F2 Gives a sorted two-element subfile of F Continue with rest of one-element subfiles

32 Binary Merge Sort Split again Merge again as before Each time, the size of the sorted subgroups doubles

33 Binary Merge Sort Last splitting gives two files each in order Last merging yields a single file, entirely in order

Quicksort Quicksort uses a divide-and-conquer strategy A recursive approach The original problem partitioned into simpler sub- problems, Each sub problem considered independently. Subdivision continues until sub problems obtained are simple enough to be solved directly

Quicksort Choose some element called a pivot Perform a sequence of exchanges so that All elements that are less than this pivot are to its left and All elements that are greater than the pivot are to its right.

Quicksort If the list has 0 or 1 elements, return. // the list is sorted Else do: Pick an element in the list to use as the pivot. Split the remaining elements into two disjoint groups: SmallerThanPivot = {all elements < pivot} LargerThanPivot = {all elements > pivot} Return the list rearranged as: Quicksort(SmallerThanPivot), pivot, Quicksort(LargerThanPivot).

Quicksort Given to sort: 75, 70, 65, 84, 98, 78, 100, 93, 55, 61, 81, 68 Select, arbitrarily, the first element, 75, as pivot. Search from right for elements <= 75, stop at first element <75 And then search from left for elements > 75, starting from pivot itself, stop at first element >=75 Swap these two elements, and then repeat this process until Right and Left point at the same location

Quicksort Example 75, 70, 65, 68, 61, 55, 100, 93, 78, 98, 81, 84 When done, swap with pivot This SPLIT operation placed pivot 75 so that all elements to the left were 75. View code for split() templatesplit() template 75 is now placed appropriately Need to sort sublists on either side of 75

Quicksort Example Need to sort (independently): 55, 70, 65, 68, 61 and 100, 93, 78, 98, 81, 84 Let pivot be 55, look from each end for values larger/smaller than 55, swap Same for 2 nd list, pivot is 100 Sort the resulting sublists in the same manner until sublist is trivial (size 0 or 1) View quicksort() recursive functionquicksort()

Quicksort Split int split (ElementType x[], int first, int last) { ElementType pivot = x[first]; int left = first, right = last; While (left < right) { while (pivot < x[right]) right--; while (left < right && x[left]<=pivot) left++; if (left < right) swap(x[left,x[right]]) } int pos = right; x[first] = x[pos]; x[pos] = pivot; return pos; }

Quicksort Note visual example of a quicksort on an array etc. …

Quicksort Performance O(log 2 n) is the average case computing time If the pivot results in sublists of approximately the same size. O(n 2 ) worst-case List already ordered, elements in reverse When Split() repetitively results, for example, in one empty sublist

Improvements to Quicksort Better method for selecting the pivot is the median-of-three rule, Select the median of the first, middle, and last elements in each sublist as the pivot. Often the list to be sorted is already partially ordered Median-of-three rule will select a pivot closer to the middle of the sublist than will the “first-element” rule.

Counting Sort Counting sort assumes that each of the n input elements is an integer in the range 0 to k When k=O(n), the sort runs in O(n) time

Counting Sort Approach Sorts keys with values over range 0..k Count number of occurrences of each key Calculate # of keys < each key Place keys in sorted location using # keys counted

Radix Sort Approach 1. Decompose key C into components C1, C2, … Cd Component d is least significant, Each component has values over range 0..k