1. Prof. Amr Goneid, AUC1 CSCE 210 Data Structures and Algorithms Prof. Amr Goneid AUC Part 8b. Sorting(2): (n log n) Algorithms

2. Prof. Amr Goneid, AUC2 Sorting(2): (n log n) Algorithms General Heap Sort Merge Sort Quick Sort

3. Prof. Amr Goneid, AUC3 1. General We examine here 3 advanced sorting algorithms: Heap Sort (based on Priority Queues) Merge Sort (based on Divide & Conquer) Quick Sort (based on Divide & Conquer) All of these algorithms have a worst case complexity of O(n log n)

4. Prof. Amr Goneid, AUC4 2. Heap Sort The heap sort is a sorting algorithm based on Priority Queues. The idea is to insert all array elements into a minimum heap, then remove the top of the heap one by one back into the array.

5. Prof. Amr Goneid, AUC5 HeapSort Algorithm V1 //To sort an array X[ ] of n elements heapsort( X[1..n ], n) { int i; PQ Heap(n); for (i = 1 to n) Heap.insert(X[i]); for (i = 1 to n) X[i] = Heap.remove(); }

6. Prof. Amr Goneid, AUC6 Analysis of HeapSort V1 Worst case cost of insertion: This happens when the data are in descending order. In this case, every new insertion will have to take the element all the way up to the root, i.e. O(h) operations. Since a complete tree has a height of O(log n), the worst case cost of inserting (n) elements into a heap is O(n log n)

7. Prof. Amr Goneid, AUC7 Analysis of HeapSort V1 Worst case cost of removal: It is now easy to see that the worst case cost of removal of an element is O(log n). Removing all elements from the heap will then cost O(n log n). Worst Case cost of HeapSort Therefore, the total worst case cost for heapsort is O(n log n)

8. Prof. Amr Goneid, AUC8 Demos http://coderaptors.com/?HeapSort

9. Prof. Amr Goneid, AUC9 Performance of Heap Sort V1 The complexity of the HeapSort is O(n log n) In-Place SortNo (uses heap array) Stable AlgorithmNo This technique is satisfactory for medium to large data sets

10. Prof. Amr Goneid, AUC10 HeapSort Algorithm V2 //To sort an array X[ ] of n elements using Heapify Algorithm heapsort (X[1..n ], n) { heap_size = n; Build-MaxHeap (X[1..n], n) for i = n downto 2 { swap(X[1], X[i]) heap_size = heap_size -1 Heapify ( X, heap_size, 1) }

11. Prof. Amr Goneid, AUC11 Analysis of HeapSort V2 Build-MaxHeap takes O(n) time Each of (n-1) calls to Heapify takes O(log n) Hence Heapsort takes: T(n) = O(n) + (n-1) O(log n) = O(n log n) Does not use extra space (In-Place algorithm)

12. Prof. Amr Goneid, AUC12 Divide & Conquer Algorithms

13. Prof. Amr Goneid, AUC13 3. MergeSort (a) Merging Definition: Combine two or more sorted sequences of data into a single sorted sequence. Formal Definition: The input is two sorted sequences, A={a 1,..., a n } and B={b 1,..., b m } The output is a single sequence, merge(A,B), which is a sorted permutation of {a 1,..., a n, b 1,..., b m }.

14. Prof. Amr Goneid, AUC14 Practical Situation 13572468 p q r Array B q+1 A1A1 A2A2 qB1B1 r B2B2 p 13572468 Array A 12345678 Copy A to B Merge back to A

15. Prof. Amr Goneid, AUC15 Merge Algorithm Merge Algorithm Merge(A,p,q,r) { copy A p..r to B p..r and set A p..r to empty while (neither B1 nor B2 empty) { compare first items of B1 & B2 remove smaller of the two from its list add to end of A } concatenate remaining list to end of A return A }

16. Prof. Amr Goneid, AUC16 Example 2911201151730 1 ij p qr B A z q+1

17. Prof. Amr Goneid, AUC17 Example 2911201151730 12 ij p qr B A z

18. Prof. Amr Goneid, AUC18 Example 2911201151730 129 ij p qr B A z

19. Prof. Amr Goneid, AUC19 Example 2911201151730 12911 ij p qr B A z

20. Prof. Amr Goneid, AUC20 Example 2911201151730 1291115 ij p qr B A z

21. Prof. Amr Goneid, AUC21 Example 2911201151730 129111517 ij p qr B A z

22. Prof. Amr Goneid, AUC22 Example 2911201151730 12911151720 ij p qr B A z

23. Prof. Amr Goneid, AUC23 Example 2911201151730 1291115172030 ij p qr B A z

24. Prof. Amr Goneid, AUC24 Worst Case Analysis |L 1 | = size of L 1, |L 2 | = size of L 2 In the worst case |L 1 | = |L 2 | = n/2 Both lists empty at about same time, so everything has to be compared. Each comparison adds one item to A so the worst case is T(n) = |A|-1 = |L1|+|L2|-1 = n-1 = O(n) comparisons.

25. Prof. Amr Goneid, AUC25 (b) MergeSort Methodology Invented by Von Neumann in 1945 Recursive Divide-And-Conquer

26. Prof. Amr Goneid, AUC26 MergeSort Methodology Divides the sequence into two subsequences of equal size, sorts the subsequences and then merges them into one sorted sequence Fast, but uses an extra space

27. Prof. Amr Goneid, AUC27 Methodology (continued) Divide: Divide n element sequence into two subsequences each of n/2 elements Conquer: Sort the two sub-arrays recursively Combine: Merge the two sorted subsequences to produce a sorted sequence of n elements

28. Prof. Amr Goneid, AUC28 Algorithm MergeSort (A, p, r) // Mergesort array A[ ] locations p..r { if (p < r) // if there are 2 or more elements { q = (p+r)/2;// Divide in the middle // Conquer both MergeSort (A, p, q); MergeSort (A, q+1, r); Merge (A, p, q, r); // Combine solutions }

29. Prof. Amr Goneid, AUC29 Merge Sort Example

30. Prof. Amr Goneid, AUC30 http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/MSort.html http://coderaptors.com/?MergeSort Merge Sort Demos

31. Prof. Amr Goneid, AUC31 Performance of MergeSort MergeSort divides the array to two halves at each level. So, the number of levels is O(log n) At each level, merge will cost O(n) Therefore, the complexity of MergeSort is O(n log n) In-Place SortNo (uses extra array) Stable AlgorithmYes This technique is satisfactory for large data sets

32. Prof. Amr Goneid, AUC32 4. QuickSort (a) Methodology Invented by Sir Tony Hoare in 1962 Recursive Divide-And-Conquer algorithm Partitions array around a Pivot, then sorts parts independently Fast, in-place sorting

33. Prof. Amr Goneid, AUC33 Methodology (continued) Divide: Array a[p..r] is rearranged into two nonempty sub- arrays a[p..q] and a[q+1..r] such that each element of the first is <= each element of the second ( the pivot index is computed as part of this process) Conquer: Sort the two sub-arrays recursively Combine: The sub-arrays are already sorted in place, i.e., a[p..r] is sorted.

34. Prof. Amr Goneid, AUC34 (b) Algorithm QuickSort (a, p, r) { if (p < r ) { q = partition(a, p, r); QuickSort(a, p,q); QuickSort(a, q+1, r); }

35. Prof. Amr Goneid, AUC35 Partitioning int partition (a, p, r) { pivot = a p ; // pivot is initially the first element i = p-1; j = r+1; while (true) { do { j--; } while (a j > pivot); do { i++; } while (a i < pivot); if(i < j) swap (a i, a j ); else return j; } // j is the location of last element in left part }

36. Prof. Amr Goneid, AUC36 Example: using 1 st element as pivot 829384 i j Pivot = 1 st element = 8 p r

37. Prof. Amr Goneid, AUC37 Pivot = 8 Partitioning 829384 ij

38. Prof. Amr Goneid, AUC38 Pivot = 8 Partitioning 829384 ij

39. Prof. Amr Goneid, AUC39 Pivot = 8 Partitioning 429388 ij

40. Prof. Amr Goneid, AUC40 Pivot = 8 Partitioning 429388 ij

41. Prof. Amr Goneid, AUC41 Pivot = 8 Partitioning 429388 ji

42. Prof. Amr Goneid, AUC42 Pivot = 8 Partitioning 429388 ij

43. Prof. Amr Goneid, AUC43 Pivot = 8 Partitioning 428398 ij

44. Prof. Amr Goneid, AUC44 Pivot = 8 Partitioning 428398 ij

45. Prof. Amr Goneid, AUC45 Pivot = 8 Partitioning 428398 ij

46. Prof. Amr Goneid, AUC46 Partitioning 428398 ij final j

47. Prof. Amr Goneid, AUC47 Partitioning 428398 Left Right q = final j p r

48. Prof. Amr Goneid, AUC48 Example (continued) 4283 3284 98 89 32 23 84 48 89 234889 Final Array

49. Prof. Amr Goneid, AUC49 http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/QSort.html http://coderaptors.com/?QuickSort Quick Sort Demo

50. Prof. Amr Goneid, AUC50 Performance of QuickSort Partitioning will cost O(n) at each level Best Case: Pivot has the middle value Quicksort divides the array to two equal halves at each level. So, the number of levels is O(log n) Therefore, the complexity of QuickSort is O(n log n) Worst Case: Pivot is the minimum (maximum) value The number of levels is O(n) Therefore, the complexity of QuickSort is O(n 2 ) In-Place SortYes Stable AlgorithmNo This technique is satisfactory for large data sets

51. Prof. Amr Goneid, AUC51 See Internet Animation Sites: For Example: http://www.cs.ubc.ca/spider/harrison/Java/sorting- demo.html http://www.cs.ubc.ca/spider/harrison/Java/sorting- demo.html http://coderaptors.com/?All_sorting_algorithms

52. Prof. Amr Goneid, AUC52 Median-of-Three Partitioning If we take the elements in the first, last and middle locations in the array and then take the element middle in value between these three, this will be the median of three. In this case, there is always at least one element below and one element above. The worst case is therefore unlikely

53. Prof. Amr Goneid, AUC53 A Better Choice: Random Pivot A better solution that avoids the worst case of O(n 2 ) is to use a randomizer to select the pivot element. Pick a random element from the sub-array (p..r) as the partition element. Do this only if (r-p) > 5 to avoid cost of the randomizer

54. Prof. Amr Goneid, AUC54 A Better Choice: Random Pivot void RQuickSort (a, p, r) { if (p < r ) { if ((r-p) > 5) { int m = rand()%(r-p+1) + p; swap (a p, a m ); } q = partition(a, p, r); RQuickSort(a, p,q); RQuickSort(a, q+1, r); }