1. 242-535 ADA: 5. Quicksort1 Objective o describe the quicksort algorithm, it's partition function, and analyse its running time under different data conditions Algorithm Design and Analysis (ADA) 242-535, Semester 1 2014-2015 5. Quicksort

2. 242-535 ADA: 5. Quicksort2 1.Quicksort 2.Partitioning Function 3.Analysis of Quicksort 4.Quicksort in PracticeOverview

3. 242-535 ADA: 5. Quicksort3 Proposed by Tony Hoare in 1962. Voted one of top 10 algorithms of 20th century in science and engineering o http://www.siam.org/pdf/news/637.pdf A divide-and-conquer algorithm. Sorts “ in place ” -- rearranges elements using only the array, as in insertion sort, but unlike merge sort which uses extra storage. Very practical (after some code tuning). 1. Quicksort

4. 242-535 ADA: 5. Quicksort4 Quicksort an n-element array: 1. Divide: Partition the array into two subarrays around a pivot x such that elements in lower subarray ≤ x ≤ elements in upper subarray. 2. Conquer: Recursively sort the two subarrays. 3. Combine: Nothing to do. Key: implementing a linear-time partitioning function Divide and conquer

5. 242-535 ADA: 5. Quicksort5 quicksort(int[] A, int left, int right) if (left < right) // If the array has 2 or more items pivot = partition (A, left, right) // recursively sort elements smaller than the pivot quicksort(A, left, pivot-1) // recursively sort elements bigger than the pivot quicksort(A, pivot+1, right)Pseudocode

6. 242-535 ADA: 5. Quicksort6 Quicksort Diagram pivot

7. 242-535 ADA: 5. Quicksort7 quicksort will stop when the subarray is 0 or 1 element big. When the subarray gets to a small size, switch over to dedicated sorting code rather than relying on recursion. quicksort is tail-recursive, a recursive behaviour which can be optimized. Fine Tuning the Code

8. 242-535 ADA: 5. Quicksort8 Tail-call optimization avoids allocating a new stack frame for a called function. o It isn't necesary because the calling function only returns the value that it gets from the called function. The most common use of this technique is for optimizing tail-recursion o the recursive function can be rewritten to use a constant amount of stack space (instead of linear) Tail-Call Optimization

9. 242-535 ADA: 5. Quicksort9 Before applying tail-call optimization: Tail-Call Graphically After applying it:

10. Pseudocode Before: int foo(int n) { if (n == 0) return A(); else { int x = B(n); return foo (x); } After: int foo(int n) { if (n == 0) return A(); else { int x = B(n); goto start of foo() code with x as argument value }

11. 242-535 ADA: 5. Quicksort11 PARTITION(A, p, q) // A[p.. q] x ← A[p] // pivot = A[p] Running time i ← p// index = O(n) for n for j ← p + 1 to q elements. if A[ j] ≤ x then i ← i + 1 // move the i boundary exchange A[i] ↔ A[ j] // switch big and small exchange A[p] ↔ A[i] return i // return index of pivot 2. Partitioning Function

12. 242-535 ADA: 5. Quicksort12 Example of partitioning scan right until find something less than the pivot scan right until find something less than the pivot

13. 242-535 ADA: 5. Quicksort13 Example of partitioning

14. 242-535 ADA: 5. Quicksort14 Example of partitioning

15. 242-535 ADA: 5. Quicksort15 Example of partitioning swap 10 and 5

16. 242-535 ADA: 5. Quicksort16 Example of partitioning resume scan right until find something less than the pivot

17. 242-535 ADA: 5. Quicksort17 Example of partitioning

18. 242-535 ADA: 5. Quicksort18 Example of partitioning

19. 242-535 ADA: 5. Quicksort19 Example of partitioning swap 13 and 3

20. 242-535 ADA: 5. Quicksort20 Example of partitioning swap 10 and 2

21. 242-535 ADA: 5. Quicksort21 Example of partitioning

22. 242-535 ADA: 5. Quicksort22 Example of partitioning j runs to the end

23. 242-535 ADA: 5. Quicksort23 Example of partitioning swap pivot and 2 so in the middle

24. 242-535 ADA: 5. Quicksort24 The analysis is quite tricky. Assume all the input elements are distinct o no duplicate values makes this code faster! o there are better partitioning algorithms when duplicate input elements exist (e.g. Hoare's original code) Let T(n) = worst-case running time on an array of n elements. 3. Analysis of Quicksort

25. 242-535 ADA: 5. Quicksort25 QUICKSORT runs very slowly when its input array is already sorted (or is reverse sorted). o almost sorted data is quite common in the real-world This is caused by the partition using the min (or max) element which means that one side of the partition will have has no elements. Therefore: T(n) = T(0) +T(n-1) + Θ(n) = Θ(1) +T(n-1) + Θ(n) = T(n-1) + Θ(n) = Θ(n 2 ) (arithmetic series) 3.1. Worst-case of quicksort no elements n-1 elements

26. 242-535 ADA: 5. Quicksort26 T(n) = T(0) +T(n-1) + cn Worst-case recursion tree

27. 242-535 ADA: 5. Quicksort27 T(n) = T(0) +T(n-1) + cn T(n) Worst-case recursion tree

28. 242-535 ADA: 5. Quicksort28 T(n) = T(0) +T(n-1) + cn cn T(0) T(n-1) Worst-case recursion tree

29. 242-535 ADA: 5. Quicksort29 T(n) = T(0) +T(n-1) + cn cn T(0) c(n-1) T(0) T(n-2) Worst-case recursion tree

30. 242-535 ADA: 5. Quicksort30 T(n) = T(0) +T(n-1) + cn cn T(0) c(n-1) T(0) T(n-2) T(0) Θ(1) Worst-case recursion tree

31. 242-535 ADA: 5. Quicksort31 T(n) = T(0) +T(n-1) + cn Worst-case recursion tree

32. 242-535 ADA: 5. Quicksort32 In the worst case, quicksort isn't any quicker than insertion sort. So why bother with quicksort? It's average case running time is very good, as we'll see. Quicksort isn't Quick?

33. 242-535 ADA: 5. Quicksort33 If we’re lucky, PARTITION splits the array evenly: T(n) = 2T(n/2) + Θ(n) = Θ( n log n ) (same as merge sort) 3.2. Best-case Analysis Case 2 of the Master Method Case 2 of the Master Method

34. 242-535 ADA: 5. Quicksort34 What if the split is always 1/10 : 9/10? T(n) = T(1/10n) + T(9/10n) + Θ(n) 3.3. Almost Best-case

35. 242-535 ADA: 5. Quicksort35 T(n) Analysis of “ almost-best ” case

36. 242-535 ADA: 5. Quicksort36 cn T(1/10n) T(9/10n) Analysis of “ almost-best ” case

37. 242-535 ADA: 5. Quicksort37 cn T(1/10n) T(9/10n) T(1/100n ) T(9/100n) T(9/100n) T(81/100n) Analysis of “ almost-best ” case

38. 242-535 ADA: 5. Quicksort38 Analysis of “ almost-best ” case

39. 242-535 ADA: 5. Quicksort39 Analysis of “ almost-best ” case short path short path long path long path cn * short path cn * long path all leaves

40. 242-535 ADA: 5. Quicksort40 Short path node value: n  (1/10)n  (1/10) 2 n ...  1  n(1/10) sp = 1  n = 10 sp // take logs  log 10 n = sp Long path node value: n  (9/10)n  (9/10) 2 n ...  1  n(9/10) lp = 1  n = (10/9) lp // take logs  log 10/9 n = lp Short and Long Path Heights sp steps lp steps

41. 242-535 ADA: 5. Quicksort41 Suppose we alternate good, bad, good, bad, good, partitions …. G(n) = 2B(n/2) + Θ(n) good B(n) = L(n – 1) + Θ(n) bad Solving: G(n) = 2( G(n/2 – 1) + Θ(n/2) ) + Θ(n) = 2G(n/2 – 1) + Θ(n) = Θ(n log n) How can we make sure we choose good partitions? 3.4. Good and Bad Good!

42. 242-535 ADA: 5. Quicksort42 IDEA : Partition around a random element. Running time is then independent of the input order. No assumptions need to be made about the input distribution. No specific input leads to the worst-case behavior. The worst case is determined only by the output of a random-number generator. Randomized Quicksort

43. 242-535 ADA: 5. Quicksort43 Quicksort is a great general-purpose sorting algorithm. o especially with a randomized pivot o Quicksort can benefit substantially from code tuning o Quicksort can be over twice as fast as merge sort Quicksort behaves well even with caching and virtual memory. 4. Quicksort in Practice

44. 242-535 ADA: 5. Quicksort44 Running time estimates: Home PC executes 10 8 compares/second. Supercomputer executes 10 12 compares/second Timing Comparisons Lesson 1. Good algorithms are better than supercomputers. Lesson 2. Great algorithms are better than good ones.