Sort Algorithms

Quadratic Sorting Quadratic Sorting--O(n2) in worst case Bubble Sort Selection Sort Insertion Sort Faster Sorgint--O(nlogn) in general Merge Sort Quick Sort The presentation illustrates two quadratic sorting algorithms: Selectionsort and Insertionsort. Before this lecture, students should know about arrays, and should have seen some motivation for sorting (such as binary search of a sorted array).

Sorting an Array of Integers The picture shows an array of six integers that we want to sort from smallest to largest The picture shows a graphical representation of an array which we will sort so that the smallest element ends up at the front, and the other elements increase to the largest at the end. The bar graph indicates the values which are in the array before sorting--for example the first element of the array contains the integer 45. [0] [1] [2] [3] [4] [5]

The Bubble Sort Algorithm 0 1 2 3 4 20 50 60 10 40 40 50 50 60 10 60 20 40 50 10 60 20 40 40 50 10 50 20 40 10 50 60 Original list After Pass 1 After Pass 2

The Bubble Sort Algorithm 0 1 2 3 4 20 40 10 50 60 20 40 10 40 20 10 40 50 60 10 20 10 20 40 50 60 After Pass 2 After Pass 3 After Pass 4

The bubble sort algorithm void bubbleSort(int a[], int count) pass  count - 1 last  pass - 1 Loop (for i from 1 to pass) Loop (for j from 0 to last) If (a[j] > a[j + 1]) Then swap a[j] and a[j = 1] End If End Loop last  last - 1 End Loop Swap Algorithm temp  a[j] a[j]  a[i + 1] a[j + 1]  temp

Analysis of Bubble Sort Suppose: count = 10 maximum pass = 9 pass comparisons 1 9 2 8 3 7 4 6 5 5 6 4 7 3 8 2 9 1 void bubbleSort(int a[], int count) pass  count – 1 last  pass – 1 Loop (for i from 1 to pass) Loop (for j from 0 to last) If (a[j] > a[j + 1]) Then swap a[j] and a[j = 1] End If End Loop last  last – 1 End Loop

Analysis of Bubble Sort Total number of comparisons 9 + 8 + 7 + 6 + 5 + 4 + 3 + 2 + 1 = ? (n-1) + (n-2) + (n-3) +…+ 3 + 2 + 1 = ? (n-1) + (n-2) + (n-3)… + 3 + 2 + 1 1 + 2 + 3 …+ (n-3) + (n-2) + (n-1) n + n + n … + n + n + n = n(n-1) Therefore, ? = n(n – 1)/2 = cn2 – cn ≈ cn2 Growth Rate: O(n2)

The Selection sort Algorithm Start by finding the smallest entry. The first sorting algorithm that we'll examine is called Selectionsort. It begins by going through the entire array and finding the smallest element. In this example, the smallest element is the number 8 at location [4] of the array. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Start by finding the smallest entry. Swap the smallest entry with the first entry. Once we have found the smallest element, that element is swapped with the first element of the array... [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Start by finding the smallest entry. Swap the smallest entry with the first entry. ...like this. The smallest element is now at the front of the array, and we have taken one small step toward producing a sorted array. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Sorted side Unsorted side Part of the array is now sorted. At this point, we can view the array as being split into two sides: To the left of the dotted line is the "sorted side", and to the right of the dotted line is the "unsorted side". Our goal is to push the dotted line forward, increasing the number of elements in the sorted side, until the entire array is sorted. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Sorted side Unsorted side Find the smallest element in the unsorted side. Each step of the Selectionsort works by finding the smallest element in the unsorted side. At this point, we would find the number 15 at location [5] in the unsorted side. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Sorted side Unsorted side Find the smallest element in the unsorted side. Swap with the front of the unsorted side. This small element is swapped with the number at the front of the unsorted side, as shown here... [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Sorted side Unsorted side We have increased the size of the sorted side by one element. ...and the effect is to increase the size of the sorted side by one element. As you can see, the sorted side always contains the smallest numbers, and those numbers are sorted from small to large. The unsorted side contains the rest of the numbers, and those numbers are in no particular order. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Sorted side Unsorted side Smallest from unsorted The process continues... Again, we find the smallest entry in the unsorted side... [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Sorted side Unsorted side Swap with front The process continues... ...and swap this element with the front of the unsorted side. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm Sorted side is bigger Sorted side Unsorted side The process continues... The sorted side now contains the three smallest elements of the array. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm The process keeps adding one more number to the sorted side. The sorted side has the smallest numbers, arranged from small to large. Sorted side Unsorted side Here is the array after increasing the sorted side to four elements. [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm We can stop when the unsorted side has just one number, since that number must be the largest number. Sorted side Unsorted side And now the sorted side has five elements. In fact, once the unsorted side is down to a single element, the sort is completed. At this point the 5 smallest elements are in the sorted side, and so the the one largest element is left in the unsorted side. We are done... [0] [1] [2] [3] [4] [5]

The Selection sort Algorithm The array is now sorted. We repeatedly selected the smallest element, and moved this element to the front of the unsorted side. ...The array is sorted. The basic algorithm is easy to state and also easy to program. [0] [1] [2] [3] [4] [5]

The selection sort algorithm void selectionSort(elemType a[], int count) Loop (for i < count – 1) // Find the smallest in unsorted portion small  i Loop (for index = i + 1 to count – 1) If (a[index] < a[small] Then small <- index End If // Swap a[i] and a[small] . . . End Loop End Loop

The Insertion sort Algorithm The Insertionsort algorithm also views the array as having a sorted side and an unsorted side. Now we'll look at another sorting method called Insertionsort. The end result will be the same: The array will be sorted from smallest to largest. But the sorting method is different. However, there are some common features. As with the Selectionsort, the Insertionsort algorithm also views the array as having a sorted side and an unsorted side, ... [0] [1] [2] [3] [4] [5]

The Insertion sort Algorithm Sorted side Unsorted side The sorted side starts with just the first element, which is not necessarily the smallest element. ...like this. However, in the Selectionsort, the sorted side always contained the smallest elements of the array. In the Insertionsort, the sorted side will be sorted from small to large, but the elements in the sorted side will not necessarily be the smallest entries of the array. Because the sorted side does not need to have the smallest entries, we can start by placing one element in the sorted side--we don't need to worry about sorting just one element. But we do need to worry about how to increase the number of elements that are in the sorted side. [0] [1] [2] [3] [4] [5]

The Insertion sort Algorithm Sorted side Unsorted side The sorted side grows by taking the front element from the unsorted side... The basic approach is to take the front element from the unsorted side... [0] [1] [2] [3] [4] [5]

The Insertion sort Algorithm Sorted side Unsorted side ...and inserting it in the place that keeps the sorted side arranged from small to large. ...and insert this element at the correct spot of the sorted side. In this example, the front element of the unsorted side is 20. So the 20 must be inserted before the number 45 which is already in the sorted side. [0] [1] [2] [3] [4] [5]

The Insertion sort Algorithm Sorted side Unsorted side In this example, the new element goes in front of the element that was already in the sorted side. After the insertion, the sorted side contains two elements. These two elements are in order from small to large, although they are not the smallest elements in the array. [0] [1] [2] [3] [4] [5]

The Insertion sort Algorithm Sorted side Unsorted side Sometimes we are lucky and the new inserted item doesn't need to move at all. Sometimes we are lucky and the newly inserted element is already in the right spot. This happens if the new element is larger than anything that's already in the array. [0] [1] [2] [3] [4] [5]

The Insertion sort Algorithm Sorted side Unsorted side Sometimes we are lucky twice in a row. Sometimes we are lucky twice in a row. [0] [1] [2] [3] [4] [5]

How to Insert One Element Copy the new element to a separate location. Sorted side Unsorted side The actual insertion process requires a bit of work that is shown here. The first step of the insertion is to make a copy of the new element. Usually this copy is stored in a local variable. It just sits off to the side, ready for us to use whenever we need it. [0] [1] [2] [3] [4] [5]

How to Insert One Element Shift elements in the sorted side, creating an open space for the new element. After we have safely made a copy of the new element, we start shifting elements from the end of the sorted side. These elements are shifted rightward, to create an "empty spot" for our new element to be placed. In this example we take the last element of the sorted side and shift it rightward one spot... [0] [1] [2] [3] [4] [5]

How to Insert One Element Shift elements in the sorted side, creating an open space for the new element. ...like this. Is this the correct spot for the new element? No, because the new element is smaller than the next element in the sorted section. So we continue shifting elements rightward... [0] [1] [2] [3] [4] [5]

How to Insert One Element Continue shifting elements... This is still not the correct spot for our new element, so we shift again... [0] [1] [2] [3] [4] [5]

How to Insert One Element Continue shifting elements... ...and shift one more time... [0] [1] [2] [3] [4] [5]

How to Insert One Element ...until you reach the location for the new element. Finally, this is the correct location for the new element. In general there are two situations that indicate the "correct location" has been found: 1. We reach the front of the array (as happened here), or 2. We reached an element that is less than or equal to the new element. [0] [1] [2] [3] [4] [5]

How to Insert One Element Copy the new element back into the array, at the correct location. Sorted side Unsorted side Once the correct spot is found, we copy the new element back into the array. The number of elements in the sorted side has increased by one. [0] [1] [2] [3] [4] [5]

How to Insert One Element The last element must also be inserted. Start by copying it... Sorted side Unsorted side The last element of the array also needs to be inserted. Start by copying it to a safe location. [0] [1] [2] [3] [4] [5]

The insertion sort algorithm void insertionSort(elemType a[], int count) Loop (for next = 1 to count – 1) insert  a[next] move  next Loop (while move > 0 && a[move – 1] > insert) // Shift to right a[move]  a[move – 1] move  move - 1 End Loop // Insert item in correct position a[move]  insert End Loop

A Quiz How many shifts will occur before we copy this element back into the array? Now we will shift elements rightward. How many shifts will be done? [0] [1] [2] [3] [4] [5]

A Quiz [0] [1] [2] [3] [4] [5] Four items are shifted. These four elements are shifted. But the 7 at location [0] is not shifted since it is smaller than the new element. [0] [1] [2] [3] [4] [5]

A Quiz [0] [1] [2] [3] [4] [5] Four items are shifted. And then the element is copied back into the array. The new element is inserted into the array. [0] [1] [2] [3] [4] [5]

Timing and Other Issues Both Selectionsort and Insertionsort have a worst-case time of O(n2), making them impractical for large arrays. But they are easy to program, easy to debug. Insertionsort also has good performance when the array is nearly sorted to begin with. But more sophisticated sorting algorithms are needed when good performance is needed in all cases for large arrays. A quick summary of these algorithms' timing properties.

Divide & conquor method O(NlogN) sorting algorithm Merge Sort Algorithm: Divide & conquor method O(NlogN) sorting algorithm Invented by John von Neumann in 1945

Growth Rates Source: http://boopathi.in/tag/algorithmpanalysis

Merge Sort Algorithm: If the list is of length 0 or 1, then it is already sorted. : Otherwise: Divide the unsorted list into two sublists of about half the size. Sort each sublist recursively by re-applying the merge sort. Merge the two sublists back into one sorted list. Visual Demo (www.wikipediacom)

If the array is of length 0 or 1, then it is already sorted. : Merge Sort 5 7 9 4 1 3 2 8 10 6 If the array is of length 0 or 1, then it is already sorted. : Otherwise: Divide array into approximately 2 subarrays and sort each list recursively. Merge the two sublists back into one sorted list. Visual Demo (www.wikipediacom) 5 7 9 4 1 3 2 8 10 6 1 4 5 7 9 2 3 6 8 10 1 2 3 4 5 6 7 8 9 10

The Merge Sort algorithm void mergeSort(elemType a[], int count) { mergeSort(a, count, 0, count – 1) } Note: the public mergeSort() calls the private mergeSort), which is a recursive function. The following slide describes the recursive mergeSort().

The Merge Sort algorithm void mergeSort(elemType a[], int count, int low, int high) // Check if low is smaller then high; if not, the array is sorted If (low < high) Then // Get the index of the element which is in the middle middle  (low + high) / 2 // Sort the left side of the array mergeSort(a, count, low, middle) // Sort the right side of the array mergeSort(a, count, middle + 1, high) // Combine them both merge(a, count, low, middle, high) End If

The Merge algorithm void merge(elemType a[], int count, int low, int middle, int high) // Helper array elemType *helper = new elemType[count]; // Copy both parts into the helper array for (int i = low; i <= high; i++) helper[i] = data[i]; int i = low; int j = middle + 1; int k = low;

The Merge algorithm (cont) // Copy smallest values from the left or the right side back // to the original array while (i <= middle && j <= high) { if (helper[i] <= helper[j]) { data[k] = helper[i]; i++; } else { data[k] = helper[j]; j++; k++;

The Merge algorithm (cont) // Copy the rest of the left side of the array into the target array while (i <= middle) { data[k] = helper[i]; k++; i++; } // helper = NULL; delete [] helper;

Quicksort pivot left subarray right subarray Algorithm Divides array into 2 parts by choosing a pivot value All items less than the pivot are placed in the left side of pivot. All items greater than the pivot are placed in the right side of pivot. Invented by Tony Hoare Visual Demo: www.wikipedia.com

Quicksort lo pivot hi a left subarray right subarray Algorithm quicksort (elemType a[], int lo, int hi) if (lo < hi) partition the array around a pivot quicksort left subarray quicksort right subarray end if

Quicksort lo pv hi a left subarray right subarray Algorithm void quicksort (elemType a[], int lo, int hi){ int pv; // pivot value if (lo < hi){ pv = partition(a, lo, hi); quicksort(a, lo, pv – 1); quicksort(a, pv + 1, hi); } }

Quicksort – Partition() int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } Quicksort – Partition() int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } lo mid hi a int partition(elemType a[], int lo, int hi){ int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(a[lo], a[mid]); pvIndex = lo; pvValue = a[lo];

Quicksort – Partition() int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } Quicksort – Partition() int Sorts::partition(int lo, int hi) { int pvValue, pvIndex, mid; mid = (lo + hi) / 2; swap(data[lo], data[mid]); pvIndex = lo; pvValue = data[lo]; for (int i = lo + 1; i <= hi; i++) { if (data[i] < pvValue) { pvIndex++; swap(data[pvIndex], data[i]); } } swap(data[lo], data[pvIndex]); return pvIndex; } for (int i = lo + 1; i <= hi; i++){ if (a[i] < pvValue){ pvIndex++; swap(a[pvIndex], a[i]); } swap(a[lo], a[pvIndex]); return pvIndex;