1. Adapted from instructor resource slides Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 1 Mergesort and Review Chapter 13 6/15/15

2. Today Any questions on project? Exams –Review questions –Easy question/fix (I added your points wrong) come see me in class. If you need a re-grade, follow instructions in syllabus. Come see me in office hours, or make an appointment Review Thursday Break More (new) Sorting Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 2

3. 3 Categories of Sorting Algorithms Selection sort –Make passes through a list –On each pass reposition correctly some element (largest or smallest)

4. Array Based Selection Sort Pseudo-Code //x[0] is reserved For i = 1 to n-1 do the following: //Find the smallest element in the sublist x[i]…x[n] Set smallPos = i and smallest = x[smallPos] For j = i + 1 to n-1 do the following: If x[j] < smallest: //smaller element found Set smallPos = j and smallest = x[smallPos] End for //No interchange smallest with x[i], first element of this sublist. Set x[smallPos] = x[i] and x[i] = smallest End for Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 4

5. In-Class Exercise #1: Selection Sort List of 9 elements: 90, 10, 80, 70, 20, 30, 50, 40, 60 Illustrate each pass… Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 5

6. Selection Sort Solution Pass 0901080702030504060 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 6 1109080702030504060 2102080709030504060 3102030709080504060 4102030409080507060 5102030405080907060 6102030405060907080 7102030405060709080 8102030405060708090

7. 7 Categories of Sorting Algorithms Exchange sort –Systematically interchange pairs of elements which are out of order –Bubble sort does this Out of order, exchange In order, do not exchange

8. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 8 Bubble Sort Algorithm 1. Initialize numCompares to n - 1 2. While numCompares != 0, do following a. Set last = 1 // location of last element in a swap b. For i = 1 to numPairs if x i > x i + 1 Swap x i and x i + 1 and set last = i c. Set numCompares = last – 1 End while

9. In-Class Exercise #2: Bubble Sort List of 9 elements: 90, 10, 80, 70, 20, 30, 50, 40, 60 Illustrate each pass… Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 9

10. Bubble Sort Solution Pass 0901080702030504060 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 10 1 8070203050406090 2107020305040608090 3102030504060708090 4102030405060708090 5102030405060708090

11. 11 Categories of Sorting Algorithms Insertion sort –Repeatedly insert a new element into an already sorted list –Note this works well with a linked list implementation All these have computing time O(n 2 )

12. Insertion Sort Pseduo Code (Instructor’s Recommendation) for j = 2 to A.length key = A[j] //Insert A[j] into the sorted sequence A[1..j-1] i = j-1 while i > 0 and A[i] > key A[i+1] = A[i] i = i-1 A[i+1] = key Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 12

13. Insertion Sort Example Pass 0524613 1254613 2245613 3245613 4124563 5123456 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 13

14. In-Class Exercise #3: Insertion Sort List of 5 elements: 9, 3, 1, 5, 2 Illustrate each pass, along with algorithm values of key, j and i… Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 14

15. Insertion Sort Solution Pass 093152keyji 139152321,0 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 15 213952132,1,0 313592543,2 412359254,3,2,1

16. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 16 Quicksort A more efficient exchange sorting scheme than bubble sort –A typical exchange involves elements that are far apart –Fewer interchanges are required to correctly position an element. Quicksort uses a divide-and-conquer strategy –A recursive approach –The original problem partitioned into simpler sub- problems, –Each sub problem considered independently. Subdivision continues until sub problems obtained are simple enough to be solved directly

17. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 17 Quicksort Choose some element called a pivot Perform a sequence of exchanges so that –All elements that are less than this pivot are to its left and –All elements that are greater than the pivot are to its right. Divides the (sub)list into two smaller sub lists, Each of which may then be sorted independently in the same way.

18. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 18 Quicksort If the list has 0 or 1 elements, return. // the list is sorted Else do: Pick an element in the list to use as the pivot. Split the remaining elements into two disjoint groups: SmallerThanPivot = {all elements < pivot} LargerThanPivot = {all elements > pivot} Return the list rearranged as: Quicksort(SmallerThanPivot), pivot, Quicksort(LargerThanPivot).

19. In-Class Exercise #4: Quicksort List of 9 elements –30,10, 80, 70, 20, 90, 50, 40, 60 Pivot is the first element Illustrate each pass Clearly denote each sublist Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 19

20. Quicksort Solution Pass 0 301080702090504060 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 20 1 1030708090504060 2102030506040709080 3102030405060708090 TO DO: How does this change if you choose the pivot as the median?

21. A heap is a binary tree with properties: 1.It is complete Each level of tree completely filled Except possibly bottom level (nodes in left most positions) 2.The key in any node dominates the keys of its children –Min-heap: Node dominates by containing a smaller key than its children –Max-heap: Node dominates by containing a larger key than its children 21 Heaps

22. 22 Implementing a Heap Use an array or vector Number the nodes from top to bottom –Number nodes on each row from left to right Store data in i th node in i th location of array (vector)

23. 23 Implementing a Heap In an array implementation children of i th node are at myArray[2*i] and myArray[2*i+1] Parent of the i th node is at myArray[i/2]

24. 24 Basic Heap Operations Construct an empty heap Check if the heap is empty Insert an item Retrieve the largest/smallest element Remove the largest/smallest element

25. 25 Basic Heap Operations Insert an item –Place new item at end of array –“Bubble” it up to the correct place –Interchange with parent so long as it is greater/less than its parent

26. 26 Basic Heap Operations Delete max/min item –Max/Min item is the root, swap with last node in tree –Delete last element –Bubble the top element down until heap property satisfied Interchange with larger of two children

27. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 27

28. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 28 Percolate Down Algorithm 1. Set c = 2 * r 2. While r <= n do following a. If c < n and myArray[c] < myArray[c + 1] Increment c by 1 b. If myArray[r] < myArray[c] i. Swap myArray[r] and myArray[c] ii. set r = c iii. Set c = 2 * c else Terminate repetition End while

29. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 29 Heapsort Given a list of numbers in an array –Stored in a complete binary tree Convert to a heap –Begin at last node not a leaf –Apply percolated down to this subtree –Continue

30. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 30 Heapsort Algorithm 1. Consider x as a complete binary tree, use heapify to convert this tree to a heap 2. for i = n down to 2 : a. Interchange x[1] and x[i] (puts largest element at end) b. Apply percolate_down to convert binary tree corresponding to sublist in x[1].. x[i-1]

31. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 31 Heapsort Now swap element 1 (root of tree) with last element –This puts largest element in correct location Use percolate down on remaining sublist –Converts from semi-heap to heap

32. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 32 Heapsort Now swap element 1 (root of tree) with last element –This puts largest element in correct location Use percolate down on remaining sublist –Converts from semi-heap to heap

33. In-Class Exercise #4: Heapsort For each step, want to draw the heap and array 30, 10, 80, 70, 20, 90, 40 Array? Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 33 1234567 30108070209040

34. Step 1: Convert to a heap Begin at the last node that is not a leaf, apply the percolate down procedure to convert to a heap the subtree rooted at this node, move to the preceding node and percolat down in that subtree and so on, working our way up the tree, until we reach the root of the given tree. (HEAPIFY) Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 34

35. Step 1 (ctd) What is the last node that is not a leaf? Apply percolate down Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 35 1234567 30109070208040

36. Step 1 (ctd) Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 36 1234567 30709010208040

37. Step 1(ctd) Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 37 1234567 90708010203040 We now have a heap!

38. Step 2: Sort and Swap The largest element is now at the root Correctly position the largest element by swapping it with the element at the end of the list and go back and sort the remaining 6 elements Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 38 1234567 90708010203040 1234567 708010203090

39. Step 2 (ctd) This is not a heap. However, since only the root changed, it is a semi-heap Use percolate down to convert to a heap Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 39

40. Step 2 (ctd) Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 40 1234567 80704010203090 1. Swap 2. Prune 1234567 30704010208090

41. Continue the pattern Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 41 1234567 70304010208090 1234567 20304010708090

42. Continue the pattern Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 42 1234567 40302010708090 1234567 10302040708090

43. Continue the pattern Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 43 1234567 30102040708090 1234567 20103040708090

44. Complete! Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 44 1234567 20103040708090 1234567 10203040708090

45. 45 Sorting Facts Sorting schemes are either … –internal -- designed for data items stored in main memory –external -- designed for data items stored in secondary memory. (Disk Drive) Previous sorting schemes were all internal sorting algorithms: –required direct access to list elements not possible for sequential files –made many passes through the list not practical for files

46. 46 Mergesort Mergesort can be used both as an internal and an external sort. A divide and conquer algorithm Basic operation in mergesort is merging, –combining two lists that have previously been sorted –resulting list is also sorted.

47. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 47 Merge Algorithm 1. Open File1 and File2 for input, File3 for output 2. Read first element x from File1 and first element y from File2 3. While neither eof File1 or eof File2 If x < y then a. Write x to File3 b. Read a new x value from File1 Otherwise a. Write y to File3 b. Read a new y from File2 End while 4. If eof File1 encountered copy rest of of File2 into File3. If eof File2 encountered, copy rest of File1 into File3

48. 48 Mergesort Algorithm

49. In-Class Exercise #6 Take File1 and File2 and produce a sorted File 3 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 49 File 1 79193347518299 File 2 1118244961 File 3

50. Mergesort Solution Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 50 File 179193347518299 File 21118244961 File 37 9 11 18 19 24 33 47 49 51 61 82 99

51. Fun Facts Most of the time spent in merging –Combining two sorted lists of size n/2 –What is the runtime of merge()? Does not sort in-place –Requires extra memory to do the merging –Then copied back into the original memory Good for external sorting –Disks are slow –Writing in long streams is more efficient 51 O(n)

52. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 52 Binary Merge Sort Given a single file Split into two files

53. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 53 Binary Merge Sort Merge first one-element "subfile" of F1 with first one-element subfile of F2 –Gives a sorted two-element subfile of F Continue with rest of one-element subfiles

54. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 54 Binary Merge Sort Split again Merge again as before Each time, the size of the sorted subgroups doubles

55. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 55 Binary Merge Sort Last splitting gives two files each in order Last merging yields a single file, entirely in order Note we always are limited to subfiles of some power of 2

56. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 56 Natural Merge Sort Allows sorted subfiles of other sizes –Number of phases can be reduced when file contains longer "runs" of ordered elements Consider file to be sorted, note in order groups

57. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 57 Natural Merge Sort Copy alternate groupings into two files –Use the sub-groupings, not a power of 2 Look for possible larger groupings

58. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 58 Natural Merge Sort Merge the corresponding sub files EOF for F2, Copy remaining groups from F1

59. Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved. 0-13-140909-3 59 Natural Merge Sort Split again, alternating groups Merge again, now two subgroups One more split, one more merge gives sort