1. Algorithms and their Applications CS2004 (2012-2013) Professor Jasna Kuljis (adapted from Dr Steve Swift) 6.1 Classic Algorithms - Sorting

2. Previously On CS2004… So far we have looked at:  Concepts of Computation and Algorithms  Comparing algorithms  Some mathematical foundation  The Big-Oh notation  Computational Complexity  Last week we also looked at data structures Classic Algorithms - SortingSlide 2

3. A Note on Laboratory Worksheets ￭ You should all have done the mathematics test at least once now ￭ You should by now have worksheet four (4) signed off ￭ Remember not to let these worksheets “pile up”  Do not leave any sheet longer than 4 weeks Classic Algorithms - SortingSlide 3

4. Classic Algorithms - Sorting ￭ Within this lecture we are going to look at three sorting algorithms in detail  Bubble Sort  Quick Sort  Radix Sort ￭ We will then look briefly at other sorting algorithms Classic Algorithms - SortingSlide 4

5. A Note of Growth Rates Classic Algorithms - SortingSlide 5 A plot of n against a number of possible T(n) The y axis is in logarithmic scale so that all the data can be viewed We are using log base 10, but usually log base 2 is used When n = 100 log(n) = 2, n = 100, nlog(n) = 200, n 2 = 10,000 and n 3 = 1,000,000

6. Why Bother With Sorting? ￭ Sorting applications are one of the most common algorithms ￭ There are implemented in  computer games, network routing software, operating systems, AI algorithms, bin packing algorithms, etc…. ￭ The sorting problem itself is easily understood  It does not require a large amount of background knowledge before you can start implementing it ￭ The algorithms are quite small and manageable  They can be implemented in a short period of time  The are relatively simple to analyse Classic Algorithms - SortingSlide 6

7. Formal Definition of Sorting ￭ The sorting problem is a mapping from x to y, where  x and y are both n length real vectors (lists and/or arrays)  x is a permutation of y (different ordering)  y i < y i+1 for all i=0,...,n-1 ￭ All of the algorithms will apply to whether we want the list sorted  from smallest to largest (ascending order), or  from largest to smallest (descending order) or ￭ The algorithms can easily be applied to integers or character vectors  Sorting whole numbers (numerical order)  Sorting names and/or addresses (alphabetic order) Classic Algorithms - SortingSlide 7

8. Bubble Sort ￭ Imagine that the items to be sorted are kept in an array held vertically.  The items with low values are “light” and bubble up to the top.  We make repeated passes over the array, from bottom to top. If two adjacent elements are out of order (the “lighter” one is below) we swap them. The effect of this operation is that on the first pass the “lightest” item rises all the way to the top. On the second pass, the second lowest key rises to the second position, and so on. Classic Algorithms - SortingSlide 8

9. Bubble Sort – Algorithm 1 INPUT: List X of n numbers (1) for i=0 to n-2 (2) for j=n-1 downto i (3) if x j < x j-1 then (4) Swap xj and xj-1 (5) End if (6) End if OUTPUT: List X sorted in ascending order Classic Algorithms - SortingSlide 9

10. Bubble Sort – Example 1 15 4444 44444 999666666 11 6999977 76 799 677777 Classic Algorithms - SortingSlide 10 44444444 66666666 15 77777 777 999 99999 11 15 s s s s s s

11. Bubble Sort - Performance ￭ How many comparisons do we need to make  in the first pass (placing the smallest element to the top)  in the second pass  in the last pass ￭ The best case (the list is already sorted)  How many passes?  How many swaps? ￭ The worst case (the list is sorted in the reverse order)  How many passes?  How many swaps? Classic Algorithms - SortingSlide 11

12. Bubble Sort: Summary ￭ If we compare pairs of adjacent elements and none are out of order  the list is sorted  if any are out of order we must have to swap them to get an ordered list ￭ Bubble sort will make passes though the list swapping any elements that are out of order ￭ After the first pass, we know that  the smallest element must be in the correct place ￭ After the second pass, we know that  the second smallest element must be in the correct place ￭ Because of this, we can shorten each successive pass of the comparison loop Classic Algorithms - SortingSlide 12

13. Slide 13 Bubble Sort Input: X - a list of n numbers (1) Let NoSwaps = False (2) While NoSwaps = False (3) Let NoSwaps = True (4) For i = n-1 downto 0 (5) If x i-1 < x i then (6) Swap x i-1 and x i (7) Let NoSwaps = False (8) End If (9) End For (10) End While Output: X – sorted (ascending) The algorithm works by running through the list of numbers, swapping pairs that are out of order The algorithm terminates when no swaps need to be made Classic Algorithms - Sorting

14. Bubble Sort – Example 2 3311 2132 1223 Classic Algorithms - SortingSlide 14 swap 3 swaps 2311 1132 3323 111 222 333 swap 2 swaps No swaps

15. Bubble Sort: Best-Case Analysis ￭ If the elements are in sorted order at the start,  the for loop will compare the adjacent pairs but will make no swaps ￭ This means that the NoSwaps variable will remain true  The While loop is only done once ￭ There are n-1 comparisons in the best case  B(n)  O(n) Classic Algorithms - SortingSlide 15

16. Bubble Sort: Worst –Case Analysis ￭ Worst case  the while loop must be done as many times as possible  each pass of the For loop must make at least one swap of the elements ￭ The number of comparisons will be: Classic Algorithms - SortingSlide 16

17. Bubble Sort: Average-Case Analysis (Part 1) ￭ Much more difficult… ￭ How many passes do we need? ￭ We can potentially stop after any of the (at most) n-1 passes of the For loop ￭ This means that we have n-1 possibilities ￭ Our average case is given by Classic Algorithms - SortingSlide 17 where C ( i ) is the work done in the first i passes

18. Bubble Sort: Average-Case Analysis (Part 2) ￭ On the first pass  we do n-1 comparisons ￭ On the second pass  we do n-2 comparisons ￭ The number of comparisons in the first i passes, in other words C(i), is given by: Classic Algorithms - SortingSlide 18

19. Bubble Sort: Average-Case Analysis (Part 3) ￭ Putting the equation for C(i) into the equation for A(n) we get: Classic Algorithms - SortingSlide 19

20. Quick Sort ￭ Quicksort is a divide and conquer algorithm ￭ Quicksort picks an element from the list as the pivot, and partitions the list into two:  Those elements smaller than the pivot value (not necessarily in order)  Those elements larger than the pivot value (not necessarily in order) ￭ Quicksort is then applied recursively on both sub-lists as long as they consist of more than 1 element Classic Algorithms - SortingSlide 20

21. Quick Sort: Example 1 4 2 6 5 3 7 1 Pivot:4 2 3 1 4 6 5 7 Pivot:2 1 2 3 4 6 5 7 Pivot:6 1 2 3 4 5 6 7 Slide 21Classic Algorithms - Sorting

22. Quick Sort: Example 2 26 33 35 29 19 12 22 33 35 2919 12 22 29351222 12 19 2229 33 35 12 19 22 26 29 33 35 pivot partition low high partition combine with pivot Slide 22Classic Algorithms - Sorting pivot

23. The Quick Sort Algorithm Recursive Algorithm QuickSort(List,First,Last) Input: List, the elements to be put into order First, the index of the first element Last, the index of the last element (1) If First < Last Then (2) Let Pivot = PivotList(List,First,Last) (3) Call QuickSort(List,First,Pivot-1) (4) Call QuickSort(List,Pivot+1,Last) (5) End If Output: List in a sorted order PivotList is defined later Slide 23Classic Algorithms - Sorting

24. The PivotList Algorithm PivotList(list,first,last) Input: List- the elements to be put into order First- the index of the first element Last- the index of the last element 1) PivotValue = list[first] 2) PivotPoint = first 3) For index = first+1 to last 4) If list[index] < PivotValue Then 5) PivotPoint=PivotPoint+1 6) Swap(list[PivotPoint],list[index]) 7) End if 8) End For 9) Swap(list[first],list[PivotPoint]) Output: PivotPoint pivot < pivot ≥ pivot unknown First Pivot pointIndexLast Slide 24Classic Algorithms - Sorting Pick the first element as the pivot Set the pivot point as the first location of the list Move through the list comparing the pivot element to the rest If an element is smaller i ncrease the pivot point Swap this element into the new pivot point location Move pivot value into correct place

25. Quick Sort: Worst-case Analysis ￭ PivotList will do n-1 comparisons, but creates one partition that has n-1 elements and the other will have no elements  When will this happen? ￭ Because we wind up just reducing the partition by one element each time, the worst case is given by: Classic Algorithms - SortingSlide 25

26. Quick Sort: Best-case Analysis ￭ PivotList creates two parts that are the same size ￭ And then all subsequent parts are the same size as the algorithm calls itself.  this can be modelled as a binary tree ￭ Summing up over the partitions we get  B(n)=O(nLog 2 (n)) Classic Algorithms - SortingSlide 26

27. Quick Sort: Average-Case Analysis Part 1 ￭ In the average case, we need to consider all of the possible places where the pivot point winds up ￭ Because for one partition of the list, there are n-1 comparisons done, and there are n ways to partition the list, we have: Classic Algorithms - SortingSlide 27

28. Quick Sort: Average-Case Analysis Part 2 ￭ Algebra can be used to simplify this recurrence relation to: ￭ This will then solve to: ￭ Therefore, A(n)  O(nLog 2 (n)) Classic Algorithms - SortingSlide 28

29. Comparison of Sorting Algorithms MethodBestAverageWorst Bubble sortO(n)O(n 2 ) Insertion sortO(n)O(n 2 ) Merge sortO(nlog 2 (n)) Quick sortO(nlog 2 (n)) O(n 2 ) Selection sortO(n 2 ) Classic Algorithms - SortingSlide 29

30. Radix Sort ￭ The Radix sort method works only on binary or integer data ￭ In a computer all numbers are represented as binary ￭ Radix sort works by using a binary bucket sort for each binary digit ￭ Integers in Java are stored using 32 bits Classic Algorithms - SortingSlide 30

31. Radix Sort – Example part 1 ￭ Consider sorting the following list  331, 454, 230, 34, 343, 45, 59, 453, 345, 231, 9 ￭ Apply the bucket sort on the last radix position ￭ After being removed from the bucket, the elements are in the following order  230, 331, 231, 343, 453, 454, 34, 45, 345, 59, 9 ￭ Apply the bucket sort on the second-to last last radix position, and the elements are put in the buckets as follows Classic Algorithms - SortingSlide 31 23 0 331 231 343 453 454 34 45 34 5 599 [0][1][2][3][4][5][6][7][8][9]

32. Radix Sort – Example part 2 ￭ After being removed from the bucket the elements are in the following order  9, 230, 331, 231, 34, 343, 45, 345, 453, 454, 59 ￭ Apply the bucket sort on the third-to last last radix position, and the elements are put in the buckets as follows ￭ After removal from the bucket the elements are in the following order  9, 34, 45, 59, 230, 231, 331, 343, 345, 453, 454 Classic Algorithms - SortingSlide 32 9230 331 231 34 343 45 345 453 454 59 [0][1][2][3][4][5][6][7][8][9] 9 34 45 59 230 231 331 343 345 453 454 [0][1][2][3][4][5][6][7][8][9]

33. Radix Sort – Part 3 ￭ This algorithm is applied on a list of binary numbers, a decade equivalent would be similar RadixSort(List,Bits) Input: List- the elements to be put into order Bits- the number of bits in each List[i] (1) For i = 0 to Bits-1 (2) Let b = 2 i (3) Bucket sort list on bit mask b (4) End If Output: List in a sorted order Slide 33Classic Algorithms - Sorting

34. Radix Sort – Part 4 ￭ The Radix sort takes O(dn) time complexity  n is the number of items  d is the maximum number of radix positions  Best, worse and average case ￭ Can be used for alphabetical sorting, i.e. strings ￭ It is very, very fast! Classic Algorithms - SortingSlide 34

35. This Weeks Laboratory ￭ This laboratory does contribute towards your overall grade… ￭ You will be conducting some experiments which test three sorting algorithms Classic Algorithms - SortingSlide 35

36. Next Lecture ￭ We will be looking at classic graph based algorithms ￭ This will be in two weeks time ￭ Next week is ASK week (effective learning week /reading week)  no lectures and no labs are scheduled in that week Classic Algorithms - SortingSlide 36