1. Deterministic Selection and Sorting Prepared by John Reif, Ph.D. Analysis of Algorithms

2. Readings Main Reading Selections: –CLR, Chapters 2, 8, 9

3. Deterministic Selection and Sorting: Selection Algorithms and Lower Bounds Sorting Algorithms and Lower Bounds

4. Problem P size n Divide into sub problems size n 1, …, n k solve these and “glue” together solutions Time to combine solutions

5. Examples 1st lecture’s mult Fast fourier transform Binary search Merge sorting

6. Selection, and Sorting on Decision Tree Model Input a, b, c

7. Time Time = (# comparisons on longest path) Binary tree with L Leaves Facts: (1) has = L – 1 internal nodes (2) max height

8. Merging 2 lists with total of n keys Input X 1 < X 2 < … < X k and Y 1 < Y 2 < … < Y n-k Output Ordered merge of two key lists

9. Goal Provably asymptotically optimal algorithm in Decision Tree Model Use this Model because it −Allows simple proofs of lower bounds −time = # comparisons so easy to bound time costs

10. Algorithm Insert Input (X 1 < X 2 < … < X k ), (Y 1 ) Case k = n – 1 Algorithm: Binary Search by Divide- and-Conquer −(1) Compare −(2) If insert Y 1 into −elseand insert Y 1 into

11. Total Comparison Cost: Since a binary tree with n = k + 1 leaves has depth >, this is optimal!

12. Case: Merging equal length lists Input −(X 1 < X 2 < … < X k ) −(Y 1 < Y 2 < … < Y n-k ) Where

13. Algorithm 1)i  1, j  1 2)while i < k and j < k do if X i < Y j then output (X i ) and set i  i + 1 else output (Y j ) and set j  j + 1 3)output remaining elements Algorithm clearly uses 2k – 1 = n – 1 comparisons

14. Lower Bound: Consider case X 1 < Y 1 < X 2 < Y 2 < …< X k < Y k any merge algorithm must compare Claim: X i with Y i for i = 1,…, k Y i with X i+1 for j = 1,…, k-1 Otherwise we could flip Y i < X i with no change)  so requires > 2k-1 = n-1 comparisons!

15. Sorting by Divide-and-Conquer AlgorithmMerge Sort Inputset S of n keys 1)Partition S into set x of keys and set Y of keys

16. Sorting by Divide-and-Conquer (cont’d) 2)Recursively compute Merge Sort 3)Merge above sequences using n-1 comparisons 4)Output merged sequence

17. Time Analysis

18. Lower Bounds on Sorting (on decision tree model) n! distinct leaves

19. Easy Approximation (via Integration)

20. A Better Bound Using Sterling Approximation

21. Selection Problems Input X 1, X 2, …, X n and index k  {1, …, n} Output X (k) = the k’th best

22. History Rev C. L. Dodge (Lewis Carol) wrote article on lawn tennis tournament in James Gazett, 1883 Felt prizes unjust because –although winner X (1) always gets 1st prize –second X (2) may not get 2nd prize

23. History (cont’d) Carol proposed his own (nonoptimal) tournament…

24. Selection of the Champion X (1) X (1) is easily determined in n-1 comparison X (1) requires n-1 comparisons x1x1

25. Selection of the Champion X (1) (cont’d) Proof –Everyone except the champion X (1) must lose at least once!

26. Selection of the Second Best X (2) Using comparisons Algorithm 1)form a balanced binary tree for tournament to find X (1) using n-1 comparisons

27. Selection of the Second Best X (2) (cont’d)

28. 1)Let Y be the set of players knocked out by champion X (1) 2)Play a tournament among the Y’s 3)output X (2) = champion of the Y’s using more comparisons

29. Lower Bounds on Finding X (2) Requires comparisons Proof −# comparisons > m 1 + m 2 + … −Where m i = # players who lost i or more matches

30. Lower Bounds on Finding X (2) (cont’d) Claim m 1 > n -1, since at end we must know X (1) as well as X (2) Claim m 2 > (# who lost to X (1) ) -1, since everyone (except X (2) ) who lost to X (1) must also have lost one more time.

31. Lower Bounds on Finding X (2) (cont’d) lemma (# who lost to X (1) ) > in worst case proof Use oracle who “fixes” results of games so that champion X (1) plays > matches

32. Lower Bounds on Finding X (2) (cont’d) Declare X i > X j if a)X i previously undefeated and X j lost at least once b)Both undefeated but X i played more matches c)Otherwise, decide consistently with previous decisions

33. Selection by Divide-and-Conquer Algorithm − Select k (X) Input − set X of n keys and index k

35. Deriving Recurrence for Time Bounds Use a sufficiently large constant c 1 (assuming d is constant) If say d = 5, T(n) < 20 c 1 n = O(n)

36. Deriving Recurrence for Time Bounds

37. Lower Bounds for Selecting X (k) Input −X = {x 1, …, x n }, index k Theorem −Every leaf of Decision Tree has depth > n - 1

38. Proof of Lower Bound for Selecting X (k) Fix a path p from root to leaf The comparisons done on p define a relation R p Let R p + = transitive closure of R p

39. Lemma If path p determines X m = X (k) then for all im either x i R p + x m or x m R p + x i Proof − Suppose x i is unrelated to x m by R p + − They can replace x i in linear order either before or after x m to violate x m =x (k)

40. Let the “key” comparison for x i be when x i is compared with x j where either 1) j=m 2) x i R p x j and x j R p + x m, or 3) x j R p x i and x m R p + x j Fact –X i has unique “key” comparison determining either x i R p + x m or x m R p + x i  So there are n-1 “key” comparisons, each distinct!

41. RADIXSORT: Linear Time Sort for RAM Assume keys over integers 1,…,n Costs O(n) time on unit cost RAM Avoids (n log n) lower bound on SORT by avoiding comparisons instead uses indexing of RAM Generalizes (in c passes) to key domains {1, …, n c }

42. Procedure RADIXSORT Input 1)for j = 1, …, n do initialize B[j] to be the empty list 2)for I=1, …, n do add I to B[x i ] 3)Let L = (i 1, i 2, …, i n ) be the concatenation of B[1],…,B[n] 4)output

43. Deterministic Selection and Sorting Prepared by John Reif, Ph.D. Analysis of Algorithms