1. Quicksort!

2. A practical sorting algorithm Quicksort  Very efficient Tight code Good cache performance Sort in place  Easy to implement  Used in older version of C++ STL sort(begin, end)  Average case performance: O(n log n)  Worst case performance: O(n 2 )

3. Quicksort Divide & Conquer  Divide into two arrays around the last element  Recursively sort each array QUICKSORT Divide around this element

4. Quicksort Divide & Conquer  Divide into two arrays around the last element  Recursively sort each array QUICKSORT Divide around this element

5. Quicksort Divide & Conquer  Divide into two arrays around the last element  Recursively sort each array QUICKSORT Exchange

6. Quicksort Divide & Conquer  Divide into two arrays around the last element  Recursively sort each array QUICKSORT

7. Partitioning the array PARTITION(A, p, r) x = A[r] i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 xij

8. Partitioning the array PARTITION(A, p, r) x = A[r] i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 xij

9. Partitioning the array PARTITION(A, p, r) x = A[r] i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 xij

10. Partitioning the array PARTITION(A, p, r) x = A[r] i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 xij

11. Partitioning the array PARTITION(A, p, r) x = A[r] i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 xij

12. Putting it together: Quicksort QUICKSORT(A, p, r) if p < r then q = PARTITION(A, p, r) QUICKSORT(A, p, q-1) QUICKSORT(A, q+1, r) QUICKSORT rpq

13. Putting it together: Quicksort QUICKSORT(A, p, r) if p < r then q = PARTITION(A, p, r) QUICKSORT(A, p, q-1) QUICKSORT(A, q+1, r) PARTITION(A, p, r) x = A[r]; i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 To sort an array A of length n: QUICKSORT(A, 1, n)

14. Running Time QUICKSORT(A, p, r) if p < r then q = PARTITION(A, p, r) QUICKSORT(A, p, q-1) QUICKSORT(A, q+1, r) PARTITION(A, p, r) x = A[r]; i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 PARTITION(A, p, r) Time O(r-p)  Outside loop: ~5 operations 0 comparisons  Inside loop: r-p comparisons c(r-p) total operations, for some c ~= 5

15. Running Time QUICKSORT(A, p, r) if p < r then q = PARTITION(A, p, r) QUICKSORT(A, p, q-1) QUICKSORT(A, q+1, r) PARTITION(A, p, r) x = A[r]; i = p–1 for j = p to r-1 if A[j] <= x then i++; exchange A[i] and A[j] exchange A[i+1] and A[r] return i+1 QUICKSORT (A, 1, n) Let T(n) be the worst-case running time, for any n Then, T(n) ≤ max {1 ≤ q ≤ n-1} ( T(q-1) + T(n-q) +  (n) ) (why?)

16. Worst-Case Running Time Sort A = [n, n-1, n-2, …, 3, 2, 1] 1.PARTITION(A, 1, n) A = [1, n-1, n-2, …, 3, 2, n] 2.PARTITION(A, 2, n)A = [1, n-1, n-2, …, 3, 2, n] 3.PARTITION(A, 2, n-1)A = [1, 2, n-2, …, 3, n-1, n] 4.PARTITION(A, 3, n-1)A = [1, 2, n-2, …, 3, n-1, n] 5.PARTITION(A, 3, n-2)A = [1, 2, 3, …, n-2, n-1, n] 6.PARTITION(A, 4, n-2) 7.PARTITION(A, 4, n-3) … n.PARTITION(A, n/2, n/2+1) Total comparisons: (n-1) + (n-2) + … + 1 =  (n 2 )

17. Worst-Case Running Time T(n) ≤ max {1 ≤ q ≤ n-1} ( T(q-1) + T(n – q) +  (n) ) Substitution Method: Guess T(n) < cn 2 T(n) ≤ max {1 ≤ q ≤ n} ( c(q – 1) 2 + c(n – q) 2 ) + dn = c×max {1 ≤ q ≤ n} ((q – 1) 2 + (n – q) 2 ) + dn  This is max for q = 1 (or, q = n) = c(n 2 – 2n + 1) + dn = cn 2 – c(2n – 1) + dn ≤ cn 2,if we pick c such that c(2n – 1) > dn

18. A randomized version of Quicksort Pick a random number in the array as a pivot RANDOMIZED-PARTITION(A, p, r) i = RANDOM(p, r) exchange A[r] and A[i] return PARTITION(A, p, r) RANDOMIZED-QUICKSORT(A, p, r) if p < r then q = RANDOMIZED-PARTITION(A, p, r) RANDOMIZED-QUICKSORT(A, p, q – 1) RANDOMIZED-QUICKSORT(A, q + 1, r) What is the expected running time?

19. Expected running time of Quicksort Observe that the running time is a constant times the number of comparison Lemma Let X be the number of comparisons ( A[j] <= x ) performed by PARTITION. Then, the running time of QUICKSORT is  (n+X) Proof There are n calls to PARTITION. Each call does a constant amount of work, plus the for loop. Each iteration of the for loop executes the comparison A[j] <= x. Therefore, running time of PARTITION is a  (# comparisons)

20. A quick review of probability Define a sample space S, which is a set of events S = { A 1, A 2, …. } Each event A has a probability P(A), such that 1.P(A) ≥ 0 2.P(S) = P(A 1 ) + P(A 2 ) + … = 1 3.P(A or B) = P(A) + P(B) More generally, for any subset T = { B 1, B 2, … } of S, P(T) = P(B 1 ) + P(B 2 ) + …

21. A quick review of probability Example: Let S be all sequences of 2 coin tosses S = {HH, HT, TH, TT} Let each head/tail occur with 50% = 0.5 probability, independent of the other coin flips Then P(HH) = P(HT) = P(TH) = P(TT) = 0.25 Let T = “no two tails occurring one after another” = { HH, HT, TH } Then, P(T) = P(HH) + P(TH) + P(TT) = 0.75

22. A quick review of probability A random variable X is a function from S to real numbers R X: S  R Example: X = “$3 for every heads” R X: {HH, HT, TH, TT}  R X(HH) = 6; X(HT) = X(TH) = 3; X(TT) = 0

23. A quick review of probability Given two random variables X and Y, The joint probability density function f(x, y) is  f(x, y) = P(X = x and Y = y) Then,  P(X = x 0 ) =  y P(X = x 0 and Y = y)  P(Y = y o ) =  x P(X = x and Y = y 0 ) The conditional probability  P(X = x | Y = y) = P(X = x and Y = y) / P(Y = y)

24. A quick review of probability Two random variables X and Y are independent, if for all values x and y, P(X = x and Y = y) = P(X = x) P(Y = y) Example: X = 1, if first coin toss is heads; 0 otherwise Y = 1, if second coin toss is heads; 0 otherwise Easy to check that X and Y are independent

25. A quick review of probability The expected value E(X) of a random variable X, is E(X) =  x x P(X = x) Going back to the coin tosses, estimate E(X) E(X) = 6 P(X = 6) + 3 P(X = 3) + 0 P(X = 0) = 6×0.25 + 3×0.5 + 0×0.25 = 1.5 + 1.5 + 0 = 3

26. A quick review of probability Linearity of expectation: E(X + Y) = E(X) + E(Y) For any function g(x), E(g(X)) =  x g(x) P(X = x) Assume X and Y are independent: E[XY] =  x  y xyP(X = x and Y = y) =  x  y P(X = x) P(Y = y) =  x P(X = x)  y P(Y = y) = E(X) E(Y)

27. A quick review of probability Define indicator variable I(A), where A is an event 1, if A occurs  I(A) = 0, otherwise Lemma: Given a sample space S, and an event A, E[ I(A) ] = P(A) Proof: E[ I(A) ] = 1×P(A) + 0×P(S – A) = P(A)

28. Back to Quicksort Lemma Let X be the number of comparisons ( A[j] <= x ) performed by PARTITION. Then, the running time of QUICKSORT is O(n+X) Define indicator random variables: X ij = I( z i is compared to z j during the course of the algorithm ) Then, the total number of comparisons performed is: X =  i=1…n-1  j=i+1…n X ij

29. Expected running time of Quicksort Running time = X =  i=1…n-1  j=i+1…n X ij Expected running time = E[X] E[X] = E [  i=1…n-1  j=i+1…n X ij ] =  i=1…n-1  j=i+1…n E[ X ij ] =  i=1…n-1  j=i+1…n P( z i is compared to z j ) We just need to estimate P( z i is compared to z j )

30. Expected running time of Quicksort During partition of z i ……z j Once a pivot p is selected  p is compared to all elements except itself  no element p smaller than pivotlarger than pivot

31. Expected running time of Quicksort Denote by Z ij = { z i,……,z j } In order to compare z i and z j First element chosen as pivot within Z ij should be z i, or z j P( z i is compared to z j ) = = P( z i or z j is first pivot from Z ij ) = P( z i is first pivot from Z ij ) + P( z j is first pivot from Z ij ) = 1/(j – i +1) + 1/(j – i + 1) = 2/(j – i + 1)

32. Expected running time of Quicksort Now we are ready to compute E[X]: E[X] =  i=1…n-1  j=i+1…n 2/(j – i + 1) Let k = j – i E[X] =  i=1…n-1  j=i+1…n 2/(k + 1) <  i=1…n  j=1…n 2/k =  i=1…n (lg n + O(1)) by CLRS A.7 = O(n lg n )

33. Selection: Find the i th number The i th order statistics is the i th smallest element 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15