1. CIS265/506 Cleveland State University – Prof. Victor Matos
Chapter 23 Sorting
CIS265/506 Cleveland State University – Prof. Victor Matos
Adapted from: Introduction to Java Programming: Comprehensive Version, Eighth Edition by Y. Daniel Liang

2. why study sorting?
Sorting is a classic subject in computer science. There are three reasons for studying sorting algorithms.
First, sorting algorithms illustrate many creative approaches to problem solving and these approaches can be applied to solve other problems.
Second, sorting algorithms are good for practicing fundamental programming techniques using selection statements, loops, methods, and arrays.
Third, sorting algorithms are excellent examples to demonstrate algorithm performance.

3. what data to sort? data to be sorted are integers,
For simplicity, this section assumes:
data to be sorted are integers,
data are sorted in ascending order, and
data are stored in an array.
The programs can be easily modified to sort other types of data, to sort in descending order, or to sort data in an ArrayList or a LinkedList.

4. Visual Resources Sorting Algorithm Animations
Animations for the visual learner available at:
Sorting Algorithm Animations
The Sound of Sorting

5. Bubble Sort
The bubble sort algorithm makes several passes through the array.
On each pass, successive neighboring pairs are compared.
If a pair is in decreasing order, its values are swapped; otherwise, the values remain unchanged.
The technique is called a bubble sort or sinking sort, because the smaller values gradually “bubble” their way to the top and the larger values sink to the bottom.
list[i]
list[i+1]

6. Bubble Sort public class BubbleSort { /** Bubble sort method */
public static void bubbleSort(int[] list) {
boolean needNextPass = true;
for (int k = 1; k < list.length && needNextPass; k++) {
// Array may be sorted and next pass not needed
needNextPass = false;
for (int i = 0; i < list.length - k; i++) {
if (list[i] > list[i + 1]) {
// Swap list[i] with list[i + 1]
int temp = list[i];
list[i] = list[i + 1];
list[i + 1] = temp;
needNextPass = true; // Next pass still needed }

7. Bubble Sort k temp if ( list[i] > list[i+1] ) swap(list, i, i+1)
K elements are in their ‘correct’ position.
temp k
list[0]
list[1]
list[i]
list[i+1]
list[n-2]
list[n-1]
if ( list[i] > list[i+1] ) swap(list, i, i+1)

8. Bubble Sort
Bubble sort time: O(n2)

9. Bubble Sort Bubble sort time: O(n2)
Best Case Scenario: Data to be sorted is already sorted ! Then it behaves as O(n)
Worst Case Scenario: Data to be sorted is sorted in the opposite order!

10. Merge Sort Algorithm
The merge sort algorithm can be described recursively as follows:
The algorithm divides the array into two halves and applies merge sort on each half recursively.
After the two halves are sorted, merge them. 10

11. Merge Sort Algorithm (1 of 2)
public class MergeSort {
/** Divide & Conquer method for sorting the numbers in given array */
public static void mergeSort(int[] list) {
if (list.length > 1) {
// Merge sort the first half
int[] firstHalf = new int[list.length / 2];
System.arraycopy(list, 0, firstHalf, 0, list.length / 2);
mergeSort(firstHalf);
// Merge sort the second half
int secondHalfLength = list.length - list.length / 2;
int[] secondHalf = new int[secondHalfLength];
System.arraycopy(list, list.length / 2,
secondHalf, 0, secondHalfLength);
mergeSort(secondHalf);
// Merge firstHalf with secondHalf
int[] temp = merge(firstHalf, secondHalf);
System.arraycopy(temp, 0, list, 0, temp.length); } 11

12. Merge Sort Algorithm (2 of 2)
/** Merge two sorted lists */
private static int[] merge(int[] list1, int[] list2) {
int[] temp = new int[list1.length + list2.length];
int current1 = 0; // Current index in list1
int current2 = 0; // Current index in list2
int current3 = 0; // Current index in temp
while (current1 < list1.length && current2 < list2.length) {
if (list1[current1] < list2[current2])
temp[current3++] = list1[current1++];
else
temp[current3++] = list2[current2++]; }
while (current1 < list1.length)
while (current2 < list2.length)
return temp;
}//merge 12

13. Merge Sort

14. Merge Two Sorted Lists

15. Merge Sort Time
Let T(n) denote the time required for sorting an array of n elements using merge sort. Without loss of generality, assume n is a power of 2. The merge sort algorithm splits the array into two subarrays, sorts the subarrays using the same algorithm recursively, and then merges the subarrays. So,

16. Merge Sort Time
The first T(n/2) is the time needed for sorting the first half of the array and the second T(n/2) is the time for sorting the second half.
To merge two subarrays, it takes
at most n-1 comparisons to compare the elements from the two subarrays, and
(b) n moves to transfer elements to the temporary array.
So, the total time is 2n-1. Therefore,

17. Merge Sort Time

18. Quick Sort QuickSort works as follows:
The algorithm selects an element, called the pivot, in the array.
Divide the array into two parts such that all the elements in the first part are less than or equal to the pivot and all the elements in the second part are greater than the pivot.
Recursively apply the quick sort algorithm to the first part and then the second part.
pivot <
> pivot
Pivot
First part
Second part

19. Quick Sort 1 of 2
public class QuickSort { // helper method public static void quickSort(int[] list) { quickSort(list, 0, list.length - 1); } private static void quickSort(int[] list, int first, int last) { if (last > first) { int pivotIndex = partition(list, first, last); quickSort(list, first, pivotIndex - 1); quickSort(list, pivotIndex + 1, last);

20. Quick Sort /** Partition the array list[first..last] */
private static int partition(int[] list, int first, int last) {
int pivot = list[first]; // Choose the first element as the pivot
int low = first + 1; // Index for forward search
int high = last; // Index for backward search
while (high > low) {
// Search forward from left
while (low <= high && list[low] <= pivot)
low++;
// Search backward from right
while (low <= high && list[high] > pivot)
high--;
// Swap two elements in the list
if (high > low) {
int temp = list[high];
list[high] = list[low];
list[low] = temp; }
while (high > first && list[high] >= pivot)
// Swap pivot with list[high]
if (pivot > list[high]) {
list[first] = list[high];
list[high] = pivot;
return high;
else {
return first;
Quick Sort

21. Quick Sort

22. Quick Sort First/Pvt: 0 Low: 2 High: 9 left
First/Pvt: 0 Low: 5 High: 4 swap pivot
First/Pvt: 6 Low: 9 High: 8 right
First/Pvt: 0 Low: 2 High: 7 right
PIVOT= 4
First/Pvt: 6 Low: 9 High: 8 swap pivot
First/Pvt: 0 Low: 1 High: 3 left
First/Pvt: 0 Low: 2 High: 7 swap
First/Pvt: 0 Low: 1 High: 0 right
PIVOT= 8
First/Pvt: 6 Low: 7 High: 6 adjust
First/Pvt: 0 Low: 4 High: 7 left
PIVOT= 0
First/Pvt: 1 Low: 3 High: 3 left
PIVOT= 6
First/Pvt: 0 Low: 4 High: 6 right
First/Pvt: 1 Low: 3 High: 2 right
First/Pvt: 0 Low: 4 High: 6 swap
First/Pvt: 1 Low: 3 High: 2 swap pivot
First/Pvt: 0 Low: 6 High: 6 left
PIVOT= 2
First/Pvt: 0 Low: 6 High: 5 adjust
First/Pvt: 6 Low: 7 High: 9 left
First/Pvt: 0 Low: 6 High: 5 swap pivot
First/Pvt: 6 Low: 7 High: 9 right
PIVOT= 5
First/Pvt: 6 Low: 7 High: 9 swap
First/Pvt: 0 Low: 5 High: 4 left
First/Pvt: 6 Low: 9 High: 9 left
First/Pvt: 0 Low: 5 High: 4 right

23. Partition

24. Quick Sort Time
To partition an array of n elements, it takes n-1 comparisons (pivot vs. all other) and n moves in the worst case. So, the time required for partition is O(n).

25. Quick Sort: Worst-Case Time
In the worst case, each time the pivot divides the array into one big subarray with the other empty. The size of the big subarray is one less than the one before it is divided. The algorithm requires O(n2) time:

26. Quick Sort: Best-Case Time
In the best case, each time the pivot divides the array into two parts of about the same size.
Let T(n) denote the time required for sorting an array of n elements using quickSort. Therefore,

27. Quick Sort: Average-Case Time
On the average, each time the pivot will not divide the array into two parts of the same size nor one empty part.
Statistically, the sizes of the two parts are very close. So the average time is O(nlogn).
The exact average-case analysis is beyond the scope of this book.

28. Heaps
Heap is a useful data structure for designing efficient sorting algorithms and priority queues.
A heap is a binary tree with the following properties:
It is a complete binary tree.
Each node is greater than or equal to any of its children.
a > b
a > c
either (b≤ c) or (c ≤ b)

29. Complete Binary Tree
A binary tree is complete if every level of the tree is full except that the last level may not be full and all the leaves on the last level are placed in left-most mode.
For example, in the figure below, the binary trees in (a) and (b) are complete, but the binary trees in (c) and (d) are not complete.
Further, the binary tree in (a) is a heap, but the binary tree in (b) is not a heap, because the root (39) is less than its right child (42).
(a) (b) (c) (d)

30. Representing a Heap Using an Array
Node position
Parent
Left Child
Right Child i
( i – 1) / 2
2 i + 1
2 i + 2
Example. The node for element 39 is at position 4, so its left child (element 14) is at ….. 9 (2*4+1), its right child (element 33) is at … 10 (2*4+2), and its parent (element 42) is at ……… 1 ((4-1)/2).

31. Adding Elements to a Heap
Adding 3, 5, 1, 19, 11, and 22 to a heap, initially empty

32. Adding Elements to a Heap
Add new node to the end of the heap
Rebuild heap as follows
Let last node be the current node;
While (the current node is greater than its parent) {
Swap the current node with its parent; }

33. Rebuild the heap after adding a new node
Adding 88 to the heap

34. Removing the Root and Rebuild the Tree
Remove the root (maximum value in the heap)
Rebuild heap as follows:
Move the last node to replace the root;
Let the new root be the current node;
While(the current node has children and
the current node is smaller than one of its children) {
Swap the current node with the larger children; }

35. Removing the Root and Rebuild the Tree
Removing root 62 from the heap

36. Removing the Root and Rebuild the Tree
Move 9 to root

37. Removing the Root and Rebuild the Tree
Swap 9 with 59

38. Removing the Root and Rebuild the Tree
Swap 9 with 44 59

39. Removing the Root and Rebuild the Tree
Swap 9 with 30 59

40. The Heap Class Note: This is NOT a class in the Collection Framework.
You need to implement your own (perhaps using an ArrayList to represent a binary tree)

41. Heap Sort public class HeapSort { /** Heap sort method */
public static <E extends Comparable> void heapSort(E[] list) {
// Create a Heap of integers
Heap<E> heap = new Heap<E>();
// Add elements to the heap
for (int i = 0; i < list.length; i++)
heap.add(list[i]);
// Remove root element from the heap
for (int i = list.length - 1; i >= 0; i--)
list[i] = heap.remove(); }
Note: Heap is not a primitive type, you must provide your own version.

42. Heap Sort Time Let h denote the height for a heap of n elements.
Since a heap is a complete binary tree,
the first level has 1 node,
the second level has 2 nodes,
the kth level has 2(k-1) nodes,
the (h-1)th level has 2(h-2) nodes, and
the hth level has at least one node and at most 2(h-1) nodes.
Therefore, the height of a Heap is O(log n).
Since the add method traces a path from a leaf to a root, it takes at most h steps to add a new element to the heap. Then it takes O(nlogn) to create a heap holding n elements.
Similar reasoning for the remove method, yielding O(nlogn).
Therefore Heap-Sort requires O(nlogn) time.

43. Bucket Sort and Radix Sort
The lower bound for general sorting algorithms is O(nlogn).
So, no sorting algorithms based on comparisons can perform better than O(nlogn).
However, if the keys are small integers, you can use bucket sort without having to compare the keys.
In general Bucket Sort complexity is the order of O(Kn) where K depends on the number of buckets and the size of the key

44. Radix Sort Uses 10 buckets and a list of input keys.
The ith digit of a key indicates the host bucket.
Begin with least significant key digit and place list items in corresponding buckets.
Remove data from buckets to reconstruct list.
Repeat with next least significant digit until all key positions have been used.
The cost of sorting is T(n) = kc n
where k is the number of digits in the largest of the n key values, and c is the card’s processing time (read, compare, move to pocket).
Therefore the complexity of Radix Sort is O(n) 44

45. Radix Sort IBM 83 Card Sorting Machine 45

46. Radix Sort
Example:
List = { 331, 454, 230, 034, 343, 045, 345, 059, 453, 345, 231, 009 }
Bucket 1 2 3 4 5 6 7 8 9
Last radix
230
331
231
343
453
454
034
045
345
059
009
Remove
230, 331, 231, 343, 453, 454, 034, 045, 345, 059, 009
Second radix
009, 230, 331, 231, 034, 343, 045, 345, 453, 454, 059
Third radix
009, 034, 045, 059, 230, 231, 331, 343, 345, 453, 454 46

47. Shell Sort
List-to-Table: Arrange the data sequence in a two-dimensional array of G columns ( begin with G ⟵ list.length/2).
Sort the columns of the array
Table-to-List: Recombine (row-wise) the table cells into a linear data sequence.
Reduce Gap Size ( G ⟵ G / 2 )
Repeat.
The effect of the algorithm is that in each step the original data sequence is progressively partially sorted.
In each cycle the algorithm works with a smaller number of columns.
In the last step, the array consists of only one column.

48. Shell Sort

49. Shell Sort
int[] a = { 2, 9, 5, 4, 8, 1, 6, 7};
int gap = a.length / 2;
while (gap > 0) {
for (int i = gap; i < a.length; i++) {
int temp = a[i];
int j = i;
while (j >= gap && a[j - gap] > temp) {
a[j] = a[j - gap];
j -= gap;
a[j] = temp; }
gap = (int) (gap / 2);

50. Shell Sort Analysis Difficult to analyze !!!
The Gap sequence suggested by its inventor Donald Shell was [1,2,4,8,16,...,2k], leading to O(n2) comparisons and exchanges in the worst case.
A different sequence [V. Pratt's] gives an algorithm in
O(n log2 n) [not as good as QuickSort’s O(n log n) ]
Pratt, V (1979). Shellsort and sorting networks (Outstanding dissertations in the computer sciences). Garland. ISBN    (This was originally presented as the author's Ph.D. thesis, Stanford University, 1971)

51. Gnome (Stupid) Sort
private static int[] gnomeSort(int[] a) {
int i = 1;
while (i < a.length) {
if (a[i] >= a[i - 1]) // move to right while
i++; // elements grow larger
else {
swap(a, i, i - 1); // swap backwards while
if (i > 1) i--; // element grow smaller }
return a;
Unique Feature: Uses only one loop! Best case: Array a[] is already sorted T(n) = n ⟶ O(n)
Worst case: Array a[] is sorted in the opposite order. For each unsorted cell a[i] we need (i-1) comparisons with left cells, accompany by (i-1) swap operations, and (i-1) comparisons with right cells to find next element out of order. Very slow!
T(n) ≈ 3( … + (n-1) )= 3n(n+1)/2 ⟶ O(n2)
 Sarbazi-Azad, Hamid (2 October 2000). "Stupid Sort: A new sorting algorithm". Newsletter (Computing Science Department, Univ. of Glasgow) (599): 4. Retrieved April 4, LINK:

52. External Sort
To sort data stored in an external file, you may first bring data to the memory, then sort it internally. However, if the file is too large, all data in the file cannot be brought to memory at one time.

53. External Sort - Phase I
Repeatedly bring data from the file to an array, sort the array using an internal sorting algorithm, and output the data from the array to a temporary file.

54. Phase II
Merge a pair of sorted segments (e.g., S1 with S2, S3 with S4, ..., and so on) into a larger sorted segment and save the new segment into a new temporary file. Continue the same process until one sorted segment results.

55. Implementing Phase II
Each merge step merges two sorted segments to form a new larger segment. The new segment doubles the number elements.
The number of segments is reduced by half after each merge step.
A segment is too large to be brought to an array in memory.
To implement a merge step,
Copy half number of segments from file f1.dat to a temporary file f2.dat.
Then merge the first remaining segment in f1.dat with the first segment in f2.dat into a temporary file named f3.dat.

56. Implementing Phase II