1. Advanced Sorting

2. Types of Sorting Algorithms
There are many, many different types of sorting algorithms, but the primary ones are:
Bubble Sort
Selection Sort
Insertion Sort
Merge Sort
Quick Sort
Heap Sort
Shell Sort
Radix Sort
Swap Sort

3. Overview For now, we’ll consider the following: Divide and Conquer
Merge Sort
Quick Sort

4. Divide and Conquer
Base Case, solve the problem directly if it is small enough
Divide the problem into two or more similar and smaller subproblems
Recursively solve the subproblems
Combine solutions to the subproblems

5. Divide and Conquer - Sort
Base case
at most one element (left ≥ right), return
Divide A into two subarrays: FirstPart, SecondPart
Two Subproblems:
sort the FirstPart
sort the SecondPart
Recursively
sort FirstPart
sort SecondPart
Combine sorted FirstPart and sorted SecondPart

6. Overview
Divide and Conquer
Merge Sort
Quick Sort

7. Merge sort An example of recursion
Idea in merge sort is to divide an array in half, sort each half, and then merge the two halves into a single sorted array. - sorting each half by recursion
Much more efficient sorting technique than bubble, insertion, and selection sorts which take O(N2) time
Merge sort is O(N*logN).
Merge sort is also fairly easy to implement. It’s conceptually easier than quicksort, Shell sort, etc.
The downside of the merge sort is that it requires an additional array in memory, equal in size to the one being sorted.

8. Merge Sort: Idea A A is sorted! Divide into two halves
FirstPart
SecondPart
Recursively sort
SecondPart
FirstPart
Merge
Divide: if the initial array A has at least two elements (nothing needs to be done if A has zero or one elements), divide A into two subarrays each containing about half of the elements of A.
Conquer: sort the two subarrays using Merge Sort.
Combine: merging the two sorted subarray into one sorted array
A is sorted!

9. Merge Sort: Algorithm Merge-Sort (A, left, right)
if left ≥ right return
else
middle ← b(left+right)/2
Merge-Sort(A, left, middle)
Merge-Sort(A, middle+1, right)
Merge(A, left, middle, right)
Recursive Call

10. Merge-Sort: Merge A: merge A: Sorted Sorted FirstPart
Sorted SecondPart A:
A[left]
A[middle]
A[right]

11. Merge-Sort: Merge Example2 3 7 8 5 15 28 30 6 10 14 1 4 5 6 6 10 14 22 3 5 15 28 L: R:
Temporary Arrays

12. Merge-Sort: Merge Example3 1 5 15 28 30 6 10 14
k=0 L: R: 3 2 15 3 7 28 8 30 1 6 4 10 5 14 6 22
i=0
j=0

13. Merge-Sort: Merge Example1 2 5 15 28 30 6 10 14
k=1 L: R: 2 3 5 3 7 15 28 8 1 6 10 4 5 14 6 22
i=0
j=1

14. Merge-Sort: Merge Example1 2 3 15 28 30 6 10 14
k=2 L: R: 2 3 7 8 1 6 4 10 5 14 22 6
i=1
j=1

15. Merge-Sort: Merge Example1 2 3 4 6 10 14
k=3 L: R: 2 3 7 8 6 1 10 4 14 5 6 22
i=2
j=1

16. Merge-Sort: Merge Example1 2 3 4 5 6 10 14
k=4 L: R: 2 3 7 8 6 1 4 10 14 5 6 22
i=2
j=2

17. Merge-Sort: Merge Example1 2 3 4 5 6 6 10 14
k=5 L: R: 2 3 7 8 6 1 10 4 5 14 22 6
i=2
j=3

18. Merge-Sort: Merge Example1 2 3 4 5 6 7 14
k=6 L: R: 2 3 7 8 6 1 4 10 5 14 22 6
i=2
j=4

19. Merge-Sort: Merge Example1 2 3 4 5 6 7 8 14
k=7 L: R: 2 3 3 5 7 15 8 28 1 6 10 4 14 5 6 22
i=3
j=4

20. Merge-Sort: Merge Example1 2 3 4 5 6 7 8
k=8 L: R: 2 3 3 5 7 15 8 28 1 6 10 4 5 14 22 6
j=4
i=4

21. Merge(A, left, middle, right)
n1 ← middle – left + 1
n2 ← right – middle
create array L[n1], R[n2]
for i ← 0 to n1-1 do L[i] ← A[left +i]
for j ← 0 to n2-1 do R[j] ← A[n1+j]
k ← i ← j ← 0
while i < n1 & j < n2
if L[i] < R[j]
A[k++] ← L[i++]
else
A[k++] ← R[j++]
while i < n1
while j < n2
n = n1+n2
Space: n
Time : cn for some constant c

22. Merge-Sort(A, 0, 7)
Divide A:

23. Merge-Sort(A, 0, 7) Merge-Sort(A, 0, 3) , divide A: 3 7 5 1 6 2 8 4

24. Merge-Sort(A, 0, 7) Merge-Sort(A, 0, 1) , divide A: 3 7 5 1 8 4 6 2 6 6 2

25. Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 0)
, base case A: 2 6

26. Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 0), return A: 6 2

27. Merge-Sort(A, 0, 7)
Merge-Sort(A, 1, 1)
, base case A: 6 2

28. Merge-Sort(A, 0, 7)
Merge-Sort(A, 1, 1), return A: 6 2

29. Merge-Sort(A, 0, 7)
Merge(A, 0, 0, 1) A:

30. Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 1), return A:

31. Merge-Sort(A, 0, 7) Merge-Sort(A, 2, 3) , divide A: 3 7 5 1 2 6 8 4 8 8 4

32. Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), base case A: 4 8

33. Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), return A: 8 4

34. Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), base case A: 8 4

35. Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), return A: 8 4

36. Merge-Sort(A, 0, 7)
Merge(A, 2, 2, 3) A:

37. Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 3), return A:

38. Merge-Sort(A, 0, 7)
Merge(A, 0, 1, 3) A:

39. Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 3), return A:

40. Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7) A:

41. Merge-Sort(A, 0, 7)
Merge (A, 4, 5, 7) A:

42. Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7), return A:

43. Merge-Sort(A, 0, 7) Merge-Sort(A, 0, 7), done! Merge(A, 0, 3, 7) A:

44. Merge-Sort Analysis Total running time: (nlogn) Total Space:  (n) cn
2 × cn/2 = cn
n/2
n/2
log n levels
4 × cn/4 = cn
n/4
n/4
n/4
n/4
n/2 × 2c = cn 2 2 2
Total: cn log n
Total running time: (nlogn)
Total Space:  (n)

45. Merge-Sort Summary Approach: divide and conquer Time Space:
Most of the work is in the merging
Total time: (n log n)
Space:
(n), more space than other sorts.
This is typically the method that you would naturally use when sorting a pile of books, CDs cards, etc.
Best Case: Ω(n ㏒ n)
Worst Case: Ο(n ㏒ n)
Running Time: Θ(n ㏒ n)
Advantage
Stable running time
Fast running time
Disadvantage
Need extra memory space for merge step

46. Overview
Divide and Conquer
Merge Sort
Quick Sort

47. Quick Sort
Quicksort is undoubtedly the most popular sorting algorithm, and for good reason:
In the majority of situations, it’s the fastest, operating in O(N*logN) time.
Quicksort was discovered by C.A.R. Hoare in 1962.
Quicksort algorithm operates by partitioning an array into two subarrays and then calling itself recursively to quicksort each of these subarrays.
There are three basic steps:
Partition the array or subarray into left (smaller keys) and right (larger keys) groups.
Call itself recursively to sort the left group.
3. Call itself recursively again to sort the right group.

48. Partitioning
Partitioning is the underlying mechanism of quicksort, but it’s also a useful operation on its own.
To partition data is to divide it into two groups, so that all the items with a key value higher than a specified amount are in one group, and all the items with a lower key value are in another.
You can easily imagine situations in which you would want to partition data.
You may want to divide your personnel records into two groups: employees who live within 15 miles of the office and those who live farther away.
Or a school administrator might want to divide students into those with grade point averages higher and lower than 3.5, so as to know who deserves to be on the Dean’s list.

49. The Partition Algorithm

50. Quick Sort Divide: Recursively sort the FirstPart and SecondPart
Pick any element p as the pivot, e.g, the first element
Partition the remaining elements into
FirstPart, which contains all elements < p
SecondPart, which contains all elements ≥ p
Recursively sort the FirstPart and SecondPart
Combine: no work is necessary since sorting
is done in place
since sorting is done in place

51. Quick Sort p pivot p p ≤ x x < p x < p p p ≤ x A: Partition
FirstPart
SecondPart p
p ≤ x
x < p
Recursive call
x < p p
p ≤ x
Sorted
FirstPart
SecondPart
Sorted

52. Quick Sort Quick-Sort(A, left, right) if left ≥ right return else
middle ← Partition(A, left, right)
Quick-Sort(A, left, middle–1 )
Quick-Sort(A, middle+1, right)
end if
Divide: partition array into 2 subarrays such that elements in the lower part <= elements in the higher part
1. If the number of elements to be sorted is 0 or 1, then return
2. Pick any element, v (this is called the pivot)
3. Partition the other elements into two disjoint sets, S1 of elements  v, and S2 of elements > v
Conquer: recursively sort the 2 subarrays
Combine: trivial since sorting is done in place
Return QuickSort (S1) followed by v followed by QuickSort (S2)

53. Partition Example A: 4 8 6 3 5 1 7 2

54. Partition Example
i=0 A: 4 8 6 3 5 1 7 2
j=1

55. Partition Example
i=0 A: 4 8 8 6 3 5 1 7 2
j=1

56. Partition Example
i=0 A: 4 8 6 6 3 5 1 7 2
j=2

57. Partition Example
i=1
i=0 A: 4 8 3 8 6 3 3 5 1 7 2
j=3

58. Partition Example
i=1 A: 4 3 6 8 5 5 1 7 2
j=4

59. Partition Example
i=1 A: 4 3 6 8 5 1 1 7 2
j=5

60. Partition Example
i=2 A: 4 3 6 1 6 8 5 1 7 2
j=5

61. Partition Example
i=2 A: 4 3 1 8 5 6 7 7 2
j=6

62. Partition Example
i=2
i=3 A: 4 3 1 8 2 5 6 7 8 2 2
j=7

63. Partition Example
i=3 A: 4 3 1 2 5 6 7 8
j=8

64. Partition Example
i=3 A: 4 2 3 1 2 4 5 6 7 8

65. Partition Example pivot in correct position A: 2 3 1 4 5 6 7 8

66. Partition(A, left, right) x ← A[left] i ← left for j ← left+1 to right
if A[j] < x then
i ← i + 1
swap(A[i], A[j])
end if
end for j
swap(A[i], A[left])
return i
n = right – left +1
Time: cn for some constant c
Space: constant

67. Quick-Sort(A, 0, 7)
Partition A: 4

68. Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 2) , partition A: 4 5 6 7 8 2 3 1 2 1 3

69. Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 0) , return , base case 4 5 6 7 8 1 2 3 1

70. Quick-Sort(A, 0, 7)
Quick-Sort(A, 1, 1)
, base case 4 1 2 3

71. Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 2), return1 3 4 2 1 3

72. Quick-Sort(A, 0, 7) Quick-Sort(A, 4, 7) Quick-Sort(A, 2, 2), return
, partition 2 1 3 4 5

73. Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7) , partition 2 1 3 4 5 6 7 8 6 6 6

74. Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7) , partition 2 1 3 4 5 6 7 8 7 7 8

75. Quick-Sort(A, 0, 7) Quick-Sort(A, 7, 7) , base case , return 2 1 3 4 56 7 8 8

76. Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7)
, return 2 1 3 4 5 6 7 8

77. Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7)
, return 2 1 3 4 5 6 8 7

78. Quick-Sort(A, 0, 7)
Quick-Sort(A, 4, 7)
, return 2 1 3 4 5 6 8 7

79. Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 7)
, done! 2 1 3 4 5 6 8 7

80. Quick-Sort: Best Case Even Partition Total time: (nlogn) cn n n/2
2 × cn/2 = cn
4 × c/4 = cn
n/3 × 3c = cn
log n levels n
n/2
n/4 3
Total time: (nlogn)

81. Quick-Sort: Worst Case
Unbalanced Partition cn n
c(n-1)
n-1
c(n-2)
n-2 3c 3
Happens only if
input is sortd
input is reversely sorted 2 2c
Total time: (n2)

82. Quick-Sort Summary Time Space Most of the work done in partitioning.
Best/Average case takes (n log(n)) time.
Worst case takes (n2) time
Space
Sorts in-place, i.e., does not require additional space
no specific input triggers worst-case behavior
the worst-case is only determined by the output of the random-number generator
Disadvantage:
Unstable in running time
Finding “pivot” element is a big issue!

83. Summary Divide and Conquer Merge-Sort Quick-Sort
Most of the work done in Merging
(n log(n)) time
(n) space
Quick-Sort
Most of the work done in partitioning
Best/Average case takes (n log(n)) time
Worst case takes (n2) time
(1) space
no specific input triggers worst-case behavior
the worst-case is only determined by the output of the random-number generator
Disadvantage:
Unstable in running time
Finding “pivot” element is a big issue!

84. Homework
What is the running time of Merge-Sort if the array is already sorted? What is the best case running time of Merge-Sort?
Demonstrate the working of Partition on sequence (13, 5, 14, 11, 16, 12, 1, 15). What is the value of i returned at the completion of Partition?