1.
Sorting Algorithms
IT12112
Lecture 07

2. What is Sorting?
Sorting: an operation that segregates items into groups according to specified criterion.
A = { }
A = { }
Arranging things into ascending or descending order is called sorting.
i.e. Sorting is the process of arranging items in order.

3. Why Sort and Examples Consider: Sorting Books in Library
Why Sort and Examples
Consider:
Sorting Books in Library
Sorting Individuals by Height (Feet and Inches)
Sorting Movies (Alphabetical)
Sorting Numbers (Sequential)

4. Types of Sorting Algorithms
Types of Sorting Algorithms
There are many, many different types of sorting algorithms, but the primary ones are:
Bubble Sort
Selection Sort
Insertion Sort
Merge Sort
Shell Sort
Heap Sort
Quick Sort
Radix Sort
Swap Sort

5.
Review of Complexity
Most of the primary sorting algorithms run on different space and time complexity.
Time Complexity is defined to be the time the computer takes to run a program (or algorithm in our case).
Space complexity is defined to be the amount of memory the computer needs to run a program.

6.
Complexity (cont.)
Complexity in general, measures the algorithms efficiency in internal factors such as the time needed to run an algorithm.
External Factors (not related to complexity):
Size of the input of the algorithm
Speed of the Computer
Quality of the Compiler

7. Some important factors that must be considered are:
Some important factors that must be considered are:
Speed : the simplest algorithms are O(N2) while more advanced ones are O(N log N). No algorithm can make less than O(N log N) comparisons between keys.
Storage : algorithms that sort in place are the best, needing memory O(N). Those are using linked list representation need extra N words of memory for references and those that work on a copy of the file needed O(2N).

8.
Simplicity : the simpler algorithms are often easiest to implement and outer perform more sophisticated once for small problem.

9. Sorting Techniques There are three major sorting techniques. Such as
Sorting Techniques
There are three major sorting techniques. Such as
Sorting by selection
Sorting by insertion
Sorting by exchange

10. Bubble Sort
Compares adjacent array elements and exchanges their values if they are out of order
Smaller values bubble up to the top of the array and larger values sink to the bottom

11. Analysis of Bubble Sort
Provides excellent performance in some cases and very poor performances in other cases
Works best when array is nearly sorted to begin with
Worst case number of comparisons is O(n*n)
Worst case number of exchanges is O(n*n)
Best case occurs when the array is already sorted
O(n) comparisons
O(1) exchanges

12. Items of Interest
Notice that only the largest value is correctly placed
All other values are still out of order
So we need to repeat this process
101 42 35 12 77 5
Largest value correctly placed

13. BubbleSort Most frequently used sorting algorithm Algorithm:
Most frequently used sorting algorithm
Algorithm:
for j=n-1 to …. O(n)
for i=0 to j ….. O(j)
if A[i] and A[i+1] are out of order, swap them
(that’s the bubble) …. O(1)
Analysis
Bubblesort is O(n^2)
Appropriate for small arrays
Appropriate for nearly sorted arrays

14. “Bubbling” All the Elements
“Bubbling” All the Elements 77 12 35 42 5
101
N - 1 5 42 12 35 77
101 42 5 35 12 77
101 42 35 5 12 77
101 42 35 12 5 77
101

15. Reducing the Number of Comparisons
Reducing the Number of Comparisons 12 35 42 77
101 5 77 12 35 42 5
101 5 42 12 35 77
101 42 5 35 12 77
101 42 35 5 12 77
101

16. Already Sorted Collections?
Already Sorted Collections?
What if the collection was already sorted?
What if only a few elements were out of place and after a couple of “bubble ups,” the collection was sorted?
We want to be able to detect this and “stop early”! 42 35 12 5 77
101

17. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 9823 45 14 6 67 33 42

18. An Animated Example www.hndit.com N 8 did_swap false to_do 7 index 198 23 45 14 6 67 33 42

19. An Animated Example www.hndit.com N 8 did_swap false to_do 7 index 198 23 45 14 6 67 33 42

20. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 123 98 45 14 6 67 33 42

21. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 2 2398 45 14 6 67 33 42

22. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 223 98 45 14 6 67 33 42

23. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 223 45 98 14 6 67 33 42

24. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 3 2345 98 14 6 67 33 42

25. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 323 45 98 14 6 67 33 42

26. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 323 45 14 98 6 67 33 42

27. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 4 2345 14 98 6 67 33 42

28. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 423 45 14 98 6 67 33 42

29. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 423 45 14 6 98 67 33 42

30. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 5 2345 14 6 98 67 33 42

31. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 523 45 14 6 98 67 33 42

32. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 523 45 14 6 67 98 33 42

33. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 6 2345 14 6 67 98 33 42

34. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 623 45 14 6 67 98 33 42

35. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 623 45 14 6 67 33 98 42

36. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 7 2345 14 6 67 33 98 42

37. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 723 45 14 6 67 33 98 42

38. An Animated Example www.hndit.com N 8 did_swap true to_do 7 index 723 45 14 6 67 33 42 98

39. After First Pass of Outer Loop
After First Pass of Outer Loop N 8
did_swap
true
to_do 7
Finished first “Bubble Up”
index 8 23 45 14 6 67 33 42 98

40. The Second “Bubble Up” www.hndit.com N 8 did_swap false to_do 6 index1 23 45 14 6 67 33 42 98

41. The Second “Bubble Up” www.hndit.com N 8 did_swap false to_do 6 index1
No Swap 23 45 14 6 67 33 42 98

42. The Second “Bubble Up” www.hndit.com N 8 did_swap false to_do 6 index2 23 45 14 6 67 33 42 98

43. The Second “Bubble Up” www.hndit.com N 8 did_swap false to_do 6 index2
Swap 23 45 14 6 67 33 42 98

44. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 223 14 45 6 67 33 42 98

45. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 323 14 45 6 67 33 42 98

46. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 323 14 45 6 67 33 42 98

47. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 323 14 6 45 67 33 42 98

48. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 423 14 6 45 67 33 42 98

49. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 4
No Swap 23 14 6 45 67 33 42 98

50. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 523 14 6 45 67 33 42 98

51. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 523 14 6 45 67 33 42 98

52. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 523 14 6 45 33 67 42 98

53. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 623 14 6 45 33 67 42 98

54. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 623 14 6 45 33 67 42 98

55. The Second “Bubble Up” www.hndit.com N 8 did_swap true to_do 6 index 623 14 6 45 33 42 67 98

56. After Second Pass of Outer Loop
After Second Pass of Outer Loop N 8
did_swap
true
to_do 6
Finished second “Bubble Up”
index 7 23 14 6 45 33 42 67 98

57. The Third “Bubble Up” www.hndit.com N 8 did_swap false to_do 5 index 123 14 6 45 33 42 67 98

58. The Third “Bubble Up” www.hndit.com N 8 did_swap false to_do 5 index 123 14 6 45 33 42 67 98

59. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 114 23 6 45 33 42 67 98

60. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 214 23 6 45 33 42 67 98

61. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 214 23 6 45 33 42 67 98

62. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 214 6 23 45 33 42 67 98

63. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 314 6 23 45 33 42 67 98

64. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 3
No Swap 14 6 23 45 33 42 67 98

65. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 414 6 23 45 33 42 67 98

66. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 414 6 23 45 33 42 67 98

67. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 414 6 23 33 45 42 67 98

68. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 514 6 23 33 45 42 67 98

69. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 514 6 23 33 45 42 67 98

70. The Third “Bubble Up” www.hndit.com N 8 did_swap true to_do 5 index 514 6 23 33 42 45 67 98

71. After Third Pass of Outer Loop
After Third Pass of Outer Loop N 8
did_swap
true
to_do 5
Finished third “Bubble Up”
index 6 14 6 23 33 42 45 67 98

72. The Fourth “Bubble Up” www.hndit.com N 8 did_swap false to_do 4 index1 14 6 23 33 42 45 67 98

73. The Fourth “Bubble Up” www.hndit.com N 8 did_swap false to_do 4 index1
Swap 14 6 23 33 42 45 67 98

74. The Fourth “Bubble Up” www.hndit.com N 8 did_swap true to_do 4 index 16 14 23 33 42 45 67 98

75. The Fourth “Bubble Up” www.hndit.com N 8 did_swap true to_do 4 index 26 14 23 33 42 45 67 98

76. The Fourth “Bubble Up” www.hndit.com N 8 did_swap true to_do 4 index 2
No Swap 6 14 23 33 42 45 67 98

77. The Fourth “Bubble Up” www.hndit.com N 8 did_swap true to_do 4 index 36 14 23 33 42 45 67 98

78. The Fourth “Bubble Up” www.hndit.com N 8 did_swap true to_do 4 index 3
No Swap 6 14 23 33 42 45 67 98

79. The Fourth “Bubble Up” www.hndit.com N 8 did_swap true to_do 4 index 46 14 23 33 42 45 67 98

80. The Fourth “Bubble Up” www.hndit.com N 8 did_swap true to_do 4 index 4
No Swap 6 14 23 33 42 45 67 98

81. After Fourth Pass of Outer Loop
After Fourth Pass of Outer Loop N 8
did_swap
true
to_do 4
Finished fourth “Bubble Up”
index 5 6 14 23 33 42 45 67 98

82. The Fifth “Bubble Up” www.hndit.com N 8 did_swap false to_do 3 index 16 14 23 33 42 45 67 98

83. The Fifth “Bubble Up” www.hndit.com N 8 did_swap false to_do 3 index 1
No Swap 6 14 23 33 42 45 67 98

84. The Fifth “Bubble Up” www.hndit.com N 8 did_swap false to_do 3 index 26 14 23 33 42 45 67 98

85. The Fifth “Bubble Up” www.hndit.com N 8 did_swap false to_do 3 index 2
No Swap 6 14 23 33 42 45 67 98

86. The Fifth “Bubble Up” www.hndit.com N 8 did_swap false to_do 3 index 36 14 23 33 42 45 67 98

87. The Fifth “Bubble Up” www.hndit.com N 8 did_swap false to_do 3 index 3
No Swap 6 14 23 33 42 45 67 98

88. After Fifth Pass of Outer Loop
After Fifth Pass of Outer Loop N 8
did_swap
false
to_do 3
Finished fifth “Bubble Up”
index 4 6 14 23 33 42 45 67 98

89. Finished “Early” www.hndit.com N 8 did_swap false to_do 3
We didn’t do any swapping, so all of the other elements must be correctly placed.
We can “skip” the last two passes of the outer loop.
index 4 6 14 23 33 42 45 67 98

90.
void bubbleSort( int a[ ] , int nElems) { int out, in; for(out=nElems-1; out>1; out--) { // outer loop (backward) for(in=0; in<out; in++) { // inner loop (forward) if( a[in] > a[in+1] ) // out of order? swap(in, in+1); } // swap them } } // end bubbleSort() void swap(int one, int two) long temp = a[one]; a[one] = a[two]; a[two] = temp;

91. void bubbleSort(int a[ ] , int nElems) { int out, in;
void bubbleSort(int a[ ] , int nElems) {
int out, in;
for(out=nElems-1; out>1; out--) // outer loop (backward)
for(in=0; in<out; in++) // inner loop (forward)
if( a[in] > a[in+1] ) // out of order?
long temp = a[in]; // swap them
a[in] = a[in+1];
a[in+1] = temp; }
} // end bubbleSort()

92.
Summary
“Bubble Up” algorithm will move largest value to its correct location (to the right)
Repeat “Bubble Up” until all elements are correctly placed:
Maximum of N-1 times
Can finish early if no swapping occurs
We reduce the number of elements we compare each time one is correctly placed

93.
Selection Sort
Selection sort is a relatively easy to understand algorithm
Sorts an array by making several passes through the array, selecting the next smallest item in the array each time and placing it where it belongs in the array
Efficiency is O(n*n)

94. Selection Sort (continued)
Selection sort is called a quadratic sort
Number of comparisons is O(n*n)
Number of exchanges is O(n)
The total number of comparisons is always
(n-1) + (n-2) = n(n-1) / 2
No average, best, or worst case —
always the same.
An O(n2) algorithm.

95.
Selection Sort
In terms of an array A, the selection sort finds the smallest element in the array and exchanges it with A[0]. Then, ignoring A[0], the sort finds the next smallest and swaps it with A[1] and so on.

96. Selection Sorting more specifically:
more specifically:
find the smallest value in the list
switch it with the value in the first position
find the next smallest value in the list
switch it with the value in the second position
repeat until all values are in their proper places 20 8 5 10 7 5 8 20 10 7 5 7 20 10 8 5 7 8 10 20 5 7 8 10 20

97. Selection Sort An example of selection sort unsorted 15 6 10 5 3 8 3 6
An example of selection sort
unsorted 15 6 10 5 3 8 3 6 10 5 15 8 3 5 10 6 15 8 3 5 6 10 15 8 3 5 6 8 15 10
sorted 3 5 6 8 10 15 97

98. Iterative Selection Sort
Iterative selection sort algorithm
//Sort the first n elements of an array
Input: array A, int n
Output:
selectionSort1(A, n){
for(index = 0; index<n-1; index++){
indexOfNextSmallest = the index of the smallest value
among A[index], A[index+1], …A[n-1]
interchange the value of A[index] and A[indexOfNextSmallest] } 98

99. Implementing iterative selection sort in Java
void selectionSort ( int a [ ] , int nElems ) {
int out, in, min;
for(out=0; out<nElems-1; out++) // outer loop
min = out; // minimum
for(in=out+1; in<nElems; in++) // inner loop
if(a[in] < a[min] )
{ // if min greater,
min = in; // we have a new min
swap(out, min); // swap them }
} // end for(out)
} // end selectionSort()
private void swap(int one, int two)
long temp = a[one];
a[one] = a[two];
a[two] = temp;

100. void selectionSort (int arr[ ] , int n) { int i , j , minIndex , tmp;
void selectionSort (int arr[ ] , int n) {
      int i , j , minIndex , tmp;    
      for (i = 0; i < n - 1; i++) {
            minIndex = i;
            for (j = i + 1; j < n; j++)
                  if (arr[j] < arr[minIndex])
                        minIndex = j;
            if (minIndex != i) {
                  tmp = arr[i];
                  arr[i] = arr[minIndex];
                  arr[minIndex] = tmp;
            }
      } }

101. Insertion Sort
Based on the technique used by card players to arrange a hand of cards
Player keeps the cards that have been picked up so far in sorted order
When the player picks up a new card, he makes room for the new card and then inserts it in its proper place

102. Insertion Sort Algorithm
For each array element from the second to the last (nextPos = 1)
Insert the element at nextPos where it belongs in the array, increasing the length of the sorted subarray by 1

103. Insertion Sort (cont’d)
To sort an array with k elements, Insertion sort requires k – 1 passes.
Example:

104. Analysis of Insertion Sort
Maximum number of comparisons is O(n*n)
In the best case, number of comparisons is O(n)
The number of shifts performed during an insertion is one less than the number of comparisons or, when the new value is the smallest so far, the same as the number of comparisons
A shift in an insertion sort requires the movement of only one item whereas in a bubble or selection sort an exchange involves a temporary item and requires the movement of three items

105. Insertion Sort Algorithm
Algorithm
Conceptually, incremental add element to sorted array or list, starting with an empty array (list).
Incremental or batch algorithm.
Analysis
In best case, input is sorted: time is O(N)
In worst case, input is reverse sorted: time is O(N2).
Average case is (loose argument) is O(N2)
Inversion: elements out of order
critical variable for determining algorithm time-cost
each swap removes exactly 1 inversion

106. Pseudo-code for Insertion Sorting
Pseudo-code for Insertion Sorting
Place ith item in proper position:
temp = data[i]
shift those elements data[j] which greater than temp to right by one position
place temp in its proper position

107. Insertion Sort (cont’d)
public void insertionSort (double arr[ ] , int n) {
for (int k = 1 ; k < n; k++)
double temp = arr [ k ];
int i = k;
while (i > 0 && arr [i-1] > temp)
arr [i] = arr [i - 1];
i --; }
arr [i] = temp;
shift to the right
It can be programmed recursively, too.

108. Insert Action: i=1 www.hndit.com temp 8 20 8 5 10 7
i = 1, first iteration 8 20 20 5 10 7
--- 8 20 5 10 7

109. Insert Action: i=2 www.hndit.com temp 5 8 20 5 10 7
i = 2, second iteration 5 8 20 20 10 7 5 8 8 20 10 7
--- 5 8 20 10 7

110. Insert Action: i=3 www.hndit.com temp 10 5 8 20 10 7
i = 3, third iteration 10 5 8 20 20 7
--- 5 8 10 20 7

111. Insert Action: i=4 www.hndit.com temp 7 5 8 10 20 7
i = 4, forth iteration 7 5 8 10 20 20 7 5 8 10 10 20 7 5 8 8 10 20
--- 5 7 8 10 20

112.
Divide and Conquer
Divide and Conquer cuts the problem in half each time, but uses the result of both halves:
cut the problem in half until the problem is trivial
solve for both halves
combine the solutions

113. Sorting (cont’d) n2 n log2 n n Time n 10 100 1000
Sorting (cont’d)
Time n2
n log2 n
When n is large enough, An2 eventually overtakes Bnlog n no matter how small A is and how large B. n n
n , ,000,000
n log2 n ,000

114. Hierarchy of Big-Oh Complexity classes in increasing order of growth:
Complexity classes in increasing order of growth:
constant time O(1)
logarithmic O(log N)
linear O(N)
loglinear O(N log N)
quadratic O(N2)
cubic O(N3)
...
exponential O(2N)
An algorithm from a lower complexity class will run much faster than one from a higher complexity class when the value of N becomes very large.

115. Comparison of Quadratic Sorts
None of the algorithms are particularly good for large arrays

116. Exercises on Sorting
1. Show the contents of the following integer array:
43, 7, 10, 23, 18, 4, 19, 5, 66, 14
when the array is sorted in ascending order using:
(a) bubble sort, (b) selection sort, (c) insertion sort.
Explain why insertion sort works well on partially sorted arrays.
3. Implement an insertion sort on an integer array that in each pass
places both the minimum and maximum elements in their proper locations.