1. Sorting Popular algorithms:
Selection sort *
Insertion sort*
Bubble sort*
Quick sort*
Comb-sort
Shell-sort
Heap sort*
Merge sort*
Counting-sort
Radix-sort
Bucket-sort
Tim-sort
Many algorithms for sorting in parallel also exist.

2. Selection Sort Sorting Algorithm #1: Selection sort
Very easy to understand and implement.
Not very efficient.

3. Selection Sort Running time? Θ(n2) – best, worst, average
// Selection sort in Java
public static void sort(int[] a){
int minPos, temp;
for (int i=0; i<=a.length-2; i++){
// Find the position of the value that belongs in position i
minPos = i;
for (int j=i+1; j<=a.length-1; j++)
if (a[j] < a[minPos])
minPos = j;
// Swap the values in positions i and min
temp = a[i];
a[i] = a[minPos];
a[minPos] = temp; }
Running time?
Θ(n2) – best, worst, average

4. Selection Sort

5. Insertion Sort Sorting Algorithm #2: Insertion sort
Also very easy to understand and implement.
Also not very efficient.

6. Insertion Sort Initial version: Running time?
public static void insertionSort(int[] a) {
int j;
for (int i=1; i<=a.length-1; i++) {
j=i;
while (j>=1) {
if (a[j] < a[j-1]) {
temp=a[j-1];
a[j-1]=a[j];
a[j]=temp; }
j=j-1;
Running time?
Θ(n2) – best, worst, average
Analysis is same as for Selection-sort

7. Insertion Sort Second version: Running time?
// This one eliminates the boolean variable
public static void insertionSort(int[] a) {
int j;
for (int i=1; i<=a.length-1; i++) {
j=i;
while ((j>=1) && (a[j]<a[j-1])) {
temp=a[j-1];
a[j-1]=a[j];
a[j]=temp;
j = j – 1; }
Running time?
Θ(n2) – worst (list in reverse order)
Θ(n) – best (list already sorted)

8. Insertion Sort More Technically, assuming the list is already sorted…
On the ith iteration of the outer loop, as i goes from 1 to a.length-1, the inner loop executes exactly 1 time.
This gives a total of n iterations of the inner loop.
Again, note that we only counted the number of iterations of the inner loop.

9. Insertion Sort Third version: Running time?
// Another slight improvement in efficiency
public static void insertionSort(int[] a) {
int j, v;
for (int i=1; i<=a.length-1; i++) {
j=i;
v = a[j];
while ((j>=1) && (v<a[j-1])) {
a[j]=a[j-1];
j=j-1; }
a[j] = v;
Running time?
Θ(n2) – worst (list in reverse order)
Θ(n) – best (list already sorted)

10. Bubble Sort Sorting Algorithm #3: Bubble sort
Also very easy to understand and implement.
Also not very efficient.
Several minor variations and enhancements are possible.

11. Bubble Sort Initial version: Running time?
public static void bubbleSort1(int[] a) {
int temp;
for (int i=1; i<=a.length-1; i++) {
for (int j=0; j<a.length-i; j++) {
if (a[j] > a[j+1]) {
temp = a[j];
a[j] = a[j+1];
a[j+1] = temp; }
Running time?
Θ(n2) – best, worst, average
Analysis is same as for Selection-sort

12. Bubble Sort Second version: (fewer bubbles) Running time?
// This version stops when a pass occurs with no swaps.
public static void bubbleSort1(int[] a) {
int i, temp;
boolean doMore;
i = 1;
doMore = true;
while ((i<=a.length-1) && (doMore)) {
doMore = false;
for (int j=0; j<a.length-i; j++)
if (a[j] > a[j+1]) {
temp = a[j];
a[j] = a[j+1];
a[j+1] = temp; }
i = i + 1;
Running time?
Θ(n2) – worst (list in reverse order)
Θ(n) – best (list already sorted)

13. Bubble Sort More Technically, assuming the list is already sorted…
On the 1st iteration of the outer loop, inner loop executes exactly n times.
The outer loops only executes 1.
Again, note that we only counted the number of iterations of the inner loop.

14. Quick Sort Sorting Algorithm #4: Quick sort
Proposed by C.A.R. Hoare in 1962.
Divide-and-conquer algorithm.
More efficient than selection, insertion, or bubble sort, on average.
Worst case is just as bad - Θ(n2)
Very practical.
The others are very practical too for the right data, i.e., almost sorted.

15. ≤ x  x x ≥ x  x Quick Sort
Divide: Partition the array into two subarrays around a pivot x such that elements in lower subarray  x  elements in upper subarray.
Conquer: Recursively sort the two subarrays.
Combine: Trivial.
Key: Linear-time partitioning subroutine.
≤ x
 x x
≥ x
 x

16. exchange A[i]  A[ j] exchange A[ p]  A[i] return i
Quick Sort
Partitioning algorithm from the book:
PARTITION(A, p, q) A[ p . . q]
x  A[ p] pivot = A[ p]
i  p
for j  p + 1 to q
do if A[ j]  x
then i  i + 1
exchange A[i]  A[ j] exchange A[ p]  A[i]
return i

17. Quick Sort 6 10 13 5 8 3 2 11 i j

18. Quick Sort 6 10 13 5 8 3 2 11 i j

19. Quick Sort 6 10 13 5 8 3 2 11 i j

20. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 i j

21. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 i j

22. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 i j

23. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 i j

24. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 i j

25. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 i j

26. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 i j

27. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 i j

28. Quick Sort 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 2 5 3 6 8 13 10 11 i

29. QUICKSORT(A, p, r)
if p < r
then q  PARTITION(A, p, r) QUICKSORT(A, p, q–1) QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)

30. Quick Sort

31. Quick Sort i j What would partition do in the worst-case? 3 5 8 10 1113 21 35 i j

32. Quick Sort – Recursion Tree
T(n) = T(0) + T(n–1) + cn
T(n)

33. Worst Case Recursion Tree
T(n) = T(0) + T(n–1) + cn cn
T(0) T(n–1)

34. Quick Sort – Recursion Tree
T(n) = T(0) + T(n–1) + cn cn
T(0) c(n–1)
T(0) T(n–2)

35. Quick Sort – Recursion Tree
T(n) = T(0) + T(n–1) + cn cn
T(0) c(n–1)
T(0) c(n–2)
T(0) …
(1)

36. Quick Sort – Recursion Tree
T(n) = T(0) + T(n–1) + cn cn
T(0) c(n–1)
T(0) c(n–2)
T(0) …
(1)

37. Quick Sort – Recursion Tree
T(n) = T(0) + T(n–1) + cn cn
(1) c(n–1)
T(n) = (n) + (n2)
= (n2)
(1) c(n–2)
h = n
(1) …
(1)

38. Quick Sort Best case analysis:
If we get lucky, partition splits the array evenly
T(n) = 2T(n/2) + Θ(n)
= Θ(nlgn) (same as Merge-Sort)
What if the split is 1/10 : 9/10?
T(n) = T(n/10) + T(n/10) + Θ(n)
= Θ(nlgn) (left as an exercise – recursion tree)
In fact, any split by a constant proportional amount will lead to Θ(nlgn)

39. Quick Sort What if alternates between best and worst cases: Solution:
L(n) = 2U(n/2) + Θ(n)
U(n) = L(n-1) + Θ(n)
Solution:
= (2L(n/2 - 1) + Θ(n/2)) + Θ(n)
= 2L(n/2 - 1) + Θ(n)
= Θ(nlgn) (left as an exercise)

40. Heap Sort
In this context, the term heap has nothing to do with memory organization!
Heap properties:
Forms an almost-complete binary tree, i.e., completely filled on all levels, except possibly the lowest, which is filled from the left up to some point.
The value at any given node is greater than the value of both it’s children (max-heap).
The root will have the largest value. 2 4 1 7 8 9 3 14 10 16

41. Heap Sort
An important operation on heaps is Max-Heapify, which pushes a value down the tree if it violates the heap property. 2 4 1 7 8 9 3 14 10 5

42. Heap Sort
In such a case, Max-Heapify will swap the value with the larger of it’s two children and then repeat. 2 4 1 7 8 9 3 14 10 5

43. Heap Sort
In such a case, Max-Heapify will swap the value with the larger of it’s two children and then repeat. 2 4 1 7 8 9 3 5 10 14

44. Heap Sort
In such a case, Max-Heapify will swap the value with the larger of it’s two children and then repeat. 2 4 1 7 5 9 3 8 10 14

45. Heap Sort
Sometimes Max-Heapify will push a value all the way to the leaf-level. 2 4 1 7 8 9 3 14 10

46. Heap Sort
Sometimes Max-Heapify will push a value all the way to the leaf-level. 2 4 1 7 8 9 3 10 14

47. Heap Sort
Sometimes Max-Heapify will push a value all the way to the leaf-level. 2 4 1 7 3 9 8 10 14

48. Heap Sort
Sometimes Max-Heapify will push a value all the way to the leaf-level. 2 3 1 7 4 9 8 10 14

49. Heap Sort 2 4 1 7 8 9 3 14 10 16 For the above: 1 2 3 4 5 6 7 8 9 101 2 3 4 5 6 7 8 9 10 11 12 16 14 2 4 1 7 8 9 3 14 10 16
For the above:
A.length = 12
A.heap-size = 10

50. Heap Sort
The Max-Heapify procedure can then be specified as:

51. Heap Sort

52. Heap Sort

53. Heap Sort The Heap-Sort algorithm works by: Building a heap
“Removing” the value at the root of the heap
Replacing the root value with the value in the highest numbered position
Re-heapifying the array, starting at the root

54. Heap Sort

55. Heap Sort

56. Heap Sort Build-Max-Heap Analysis:
One call to Max-Heapify costs O(lgn) time.
Build-Max-Heap makes O(n) such calls.
Total is O(nlgn) cost.
This gives an upper-bound, but one that is not tight.

57. Heap Sort

58. Heap Sort

59. Heap Sort 15

60. How to Compare Algorithms in Efficiency
Empirical Analysis:
Experimentation:
Wall-clock time
CPU time
Requires many different inputs
Can you predict performance before implementing the algorithm?
Theoretical Analysis:
Approximation by counting important operations
Mathematical functions based on input size (N)

61. How Fast/Slow Can It Get? (10G Hz, assume 1010 operations/sec)
Nlog2N N2 2N 10 33
100
1,024
(10-8 sec)
664
10,000
1.3 x 1030
(4 x1012 years)
1,000
9,966
1,000,000
Forever??
132,877
100,000,000
Eternity??