1. Updated 29.3.2004

2. QuickSort

3. Problem From a given set of n integers, find the missing integer from 0 to n using O(n) queries of type: “what is bit[j] in A[i]?” Note - there are a total of nlogn bits, so we are not allowed to read the entire input!

4. Solution Ask all the n integers what their last bit is and see whether 0 or 1 is the bit which occurs less often than it is supposed to. That is the last bit of the missing integer! How can we determine the second-to-last bit?

5. Solution Ask the n/2 numbers which ended with the correct last bit! By analyzing the bit patterns of the numbers from 0 to n which end with this bit. By recurring on the remaining candidate numbers, we get the answer in T(n) = T(n/2) + n =O(n), by the Master Theorem

6. Why is sorting so important? Most of the interesting concepts in the course can be taught in the context of sorting, such as: –Divide and conquer –Randomized algorithms –Lower bounds

7. Why is sorting so important? One of the reasons why sorting is so important, is that after sorting items, other problems become very simple to solve.

8. Searching Binary search runs on a sorted set in O(logn) time Searching for an element in a non sorted set take linear time This is probably the most important application of sorting

9. Element Uniqueness Given a set of numbers we want to check if all numbers are unique. Sort the elements and linearly scan all adjacent pairs.

10. Closest pairs Given n numbers, find the pair which are closest to each other. After sorting the elements, the closest pairs will be next to each other, so a linear scan will do. Related problems….

11. Frequency distribution Which element appears the largest number of times in a set. After sorting, a linear scan will do.

12. Median and Order statistics What is the median of a set of numbers? What is the k-th largest element? After sorting the elements the k-th largest element can be found at index k, in constant time

13. Convex hulls Given n points in two dimensions, find the smallest area polygon which contains them all.

14. Huffman Codes If you are trying to minimize the size of a text file, you should want to assign different lengths to represent different characters, according to the frequency in which each character appears in the text.

15. Quicksort Although mergesort is O(nlogn), it is not convenient to be used with arrays, since it requires extra space. In practice, Quicksort is the fastest algorithm and it uses partition as its main idea.

16. Quicksort Partitioning places all the elements less than the pivot in the left part of the array, and all elements greater than the pivot in the right part of the array. The pivot fits in the slot between them.

17. Partition Example – use 10 as a pivot Note that the pivot element ends up in the correct place in the total order! Before1712619238510 After6851023191217

18. Partition First we must select a pivot element Once we have selected a pivot element, we can partition the array in one linear scan, by maintaining three sections of the array: –All elements smaller than the pivot –All elements greater than the pivot –All unexplored elements

19. Example: pivot element is 10 | 17 12 6 19 23 8 5 | 10 | 5 12 6 19 23 8 | 17 5 | 12 6 19 23 8 | 17 5 | 8 6 19 23 | 12 17 5 8 | 6 19 23 | 12 17 5 8 6 | 19 23 | 12 17 5 8 6 | 19 | 23 12 17 5 8 6 ||19 23 12 17 5 8 6 10 23 12 17 19

20. Quicksort Partition does at most n swaps and takes linear time. –The pivot element ends up in the position it retains in the final sorted order. –After a partitioning, no element flops to the other side of the pivot in the final sorted order. –Thus we can sort the elements to the left of the pivot and the right of the pivot independently! And recursively

21. QuickSort QuickSort(A, p,r) if (p < r) then q  Partition(A,p,r) QuickSort(A,p,q) QuickSort(A,q+1,r) QuickSort(A,1,length[A])

22. QuickSort public void sort (Comparable[] values) { sort (values, 0, values.length - 1); } private void sort (Comparable[] values, int from, int to) { if (from < to) { int pivot = partition (values, from, to); sort(values, from, pivot); sort(values, pivot + 1, to); }

23. private int partition (Comparable[] values, int from, int to) { Comparable pivot = values[from]; int j = to + 1; int i = from - 1; while (true) { do { j--; } while (values[j].compareTo(pivot) >= 0); do { i++; } while (values[i].compareTo(pivot) < 0); if (i < j) { Comparable temp = values[i]; values[i] = values[j]; values[j] = temp; } else { return j; }

24. Partition The partition method returns the index separating the array, but also has a side effect, which is swapping the elements in the array according to their size 7222311516 ij

25. Partition 7222311516 ij 1222371516 ij

26. Partition 1222371516 ij 1232271516 ij 1232271516 ji

27. Partition – version 2 public int partition (int[] values, int from, int to) { int pivot = values[from]; int leftWall = from; for (int i = from + 1; i <= to; i++) { if (values[i] < pivot) { leftWall++; int temp = values[i]; values[i] = values[leftWall]; values[leftWall] = temp; } int temp = values[from]; values[from] = values[leftWall]; values[leftWall] = temp; return leftWall; }

28. Partition 7222311516 lwi 7222311516 lw

29. Partition 7222311516 lwi 7232211516 lwi

30. Partition 7232211516 lwi 7231221516 lwi 1237221516 lwi

31. Partition (A[], left, right) 1. pivot  left 2. temp  right 3. while temp <> pivot 4. if A[min(pivot,temp] > A[max(pivot,temp)] 5.swap (A[pivot],A[temp]) 6. swap (pivot,temp) 7. if temp > pivot 8. temp— 9. else 10.temp++ 11. return pivot

32. Example 4536997821673408832 36997821673408845 3236997821673408845 3234597821673408869 3234597821673408869 3234097821673458869 3234045821673978869 3234045821673978869 3234016824573978869 3234016458273978869

33. Time Analysis The running time for quick sort depends on how equally partition divides the array. The chosen pivot element determines how equally partition divides the array. If partition results in equal sub arrays quicksort can be as good as merge sort If partition to equally divide the array, quicksort may be asymptotically as worse as insertion sort.

34. Best case Since each element ultimately ends up in the correct position, the algorithm correctly sorts. But how long does it take? The best case for divide-and-conquer algorithms comes when we split the input as evenly as possible. Thus in the best case, each subproblem is of size n/2.

35. Best case The partition step on each subproblem is linear in its size. Thus the total effort in partitioning each step is O(n). The total partitioning on each level is O(n), and it take log(n) levels of perfect partitions to get to single element. The total effort is therefore O(nlogn)

36. Best case

37. Worst case

38. Worst case If the pivot is the biggest or smallest element in the array, the sub-problems will be divided into size 0 and n-1, thus instead of log(n) recursive steps we end up with O(n) recursive steps and a total sorting time of O(n^2)

39. Worst case The worst case input for quick sort depends on the way we choose the pivot element. If we choose the first or last element as the pivot, the worst case is when the elements are already sorted!!

40. Worst case Having the worst case occur in a sorted array is bad, since this is an expected case in many applications. (Insertion sort deals with sorted arrays in linear time.) To eliminate this problem, pick a better pivot: 1.Use a random element of the array as the pivot. 2.take the median of three elements (first, last, middle) as the pivot. 3.The worst case remains, however, because the worst case is no longer a natural order it is much more difficult to occur.

41. Randomization Quicksort is good on average, but bad on certain worst-case instances. Suppose you picked the pivot element at random. Your enemy select a worst case input because you would have the same probability for a good pivot! By either picking a random pivot or scrambling the permutation before sorting it, we can say: ``With high probability, randomized quicksort runs in O(nlogn) time.''

42. Time Analysis Worst Case

43. Time Analysis Best Case

44. Time Analysis Average Case

45. Time Analysis