Deterministic Selection and Sorting Prepared by John Reif, Ph.D. Analysis of Algorithms

Readings Main Reading Selections: –CLR, Chapters 2, 8, 9

Deterministic Selection and Sorting: Selection Algorithms and Lower Bounds Sorting Algorithms and Lower Bounds

Problem P size n Divide into sub problems size n 1, …, n k solve these and “glue” together solutions Time to combine solutions

Examples 1st lecture’s mult Fast fourier transform Binary search Merge sorting

Selection, and Sorting on Decision Tree Model Input a, b, c

Time Time = (# comparisons on longest path) Binary tree with L Leaves Facts: (1) has = L – 1 internal nodes (2) max height

Merging 2 lists with total of n keys Input X 1 < X 2 < … < X k and Y 1 < Y 2 < … < Y n-k Output Ordered merge of two key lists

Goal Provably asymptotically optimal algorithm in Decision Tree Model Use this Model because it −Allows simple proofs of lower bounds −time = # comparisons so easy to bound time costs

Algorithm Insert Input (X 1 < X 2 < … < X k ), (Y 1 ) Case k = n – 1 Algorithm: Binary Search by Divide- and-Conquer −(1) Compare −(2) If insert Y 1 into −elseand insert Y 1 into

Total Comparison Cost: Since a binary tree with n = k + 1 leaves has depth >, this is optimal!

Case: Merging equal length lists Input −(X 1 < X 2 < … < X k ) −(Y 1 < Y 2 < … < Y n-k ) Where

Algorithm 1)i  1, j  1 2)while i < k and j < k do if X i < Y j then output (X i ) and set i  i + 1 else output (Y j ) and set j  j + 1 3)output remaining elements Algorithm clearly uses 2k – 1 = n – 1 comparisons

Lower Bound: Consider case X 1 < Y 1 < X 2 < Y 2 < …< X k < Y k any merge algorithm must compare Claim: X i with Y i for i = 1,…, k Y i with X i+1 for j = 1,…, k-1 Otherwise we could flip Y i < X i with no change)  so requires > 2k-1 = n-1 comparisons!

Sorting by Divide-and-Conquer AlgorithmMerge Sort Inputset S of n keys 1)Partition S into set x of keys and set Y of keys

Sorting by Divide-and-Conquer (cont’d) 2)Recursively compute Merge Sort 3)Merge above sequences using n-1 comparisons 4)Output merged sequence

Time Analysis

Lower Bounds on Sorting (on decision tree model) n! distinct leaves

Easy Approximation (via Integration)

A Better Bound Using Sterling Approximation

Selection Problems Input X 1, X 2, …, X n and index k  {1, …, n} Output X (k) = the k’th best

History Rev C. L. Dodge (Lewis Carol) wrote article on lawn tennis tournament in James Gazett, 1883 Felt prizes unjust because –although winner X (1) always gets 1st prize –second X (2) may not get 2nd prize

History (cont’d) Carol proposed his own (nonoptimal) tournament…

Selection of the Champion X (1) X (1) is easily determined in n-1 comparison X (1) requires n-1 comparisons x1x1

Selection of the Champion X (1) (cont’d) Proof –Everyone except the champion X (1) must lose at least once!

Selection of the Second Best X (2) Using comparisons Algorithm 1)form a balanced binary tree for tournament to find X (1) using n-1 comparisons

Selection of the Second Best X (2) (cont’d)

1)Let Y be the set of players knocked out by champion X (1) 2)Play a tournament among the Y’s 3)output X (2) = champion of the Y’s using more comparisons

Lower Bounds on Finding X (2) Requires comparisons Proof −# comparisons > m 1 + m 2 + … −Where m i = # players who lost i or more matches

Lower Bounds on Finding X (2) (cont’d) Claim m 1 > n -1, since at end we must know X (1) as well as X (2) Claim m 2 > (# who lost to X (1) ) -1, since everyone (except X (2) ) who lost to X (1) must also have lost one more time.

Lower Bounds on Finding X (2) (cont’d) lemma (# who lost to X (1) ) > in worst case proof Use oracle who “fixes” results of games so that champion X (1) plays > matches

Lower Bounds on Finding X (2) (cont’d) Declare X i > X j if a)X i previously undefeated and X j lost at least once b)Both undefeated but X i played more matches c)Otherwise, decide consistently with previous decisions

Selection by Divide-and-Conquer Algorithm − Select k (X) Input − set X of n keys and index k

Deriving Recurrence for Time Bounds Use a sufficiently large constant c 1 (assuming d is constant) If say d = 5, T(n) < 20 c 1 n = O(n)

Deriving Recurrence for Time Bounds

Lower Bounds for Selecting X (k) Input −X = {x 1, …, x n }, index k Theorem −Every leaf of Decision Tree has depth > n - 1

Proof of Lower Bound for Selecting X (k) Fix a path p from root to leaf The comparisons done on p define a relation R p Let R p + = transitive closure of R p

Lemma If path p determines X m = X (k) then for all im either x i R p + x m or x m R p + x i Proof − Suppose x i is unrelated to x m by R p + − They can replace x i in linear order either before or after x m to violate x m =x (k)

Let the “key” comparison for x i be when x i is compared with x j where either 1) j=m 2) x i R p x j and x j R p + x m, or 3) x j R p x i and x m R p + x j Fact –X i has unique “key” comparison determining either x i R p + x m or x m R p + x i  So there are n-1 “key” comparisons, each distinct!

RADIXSORT: Linear Time Sort for RAM Assume keys over integers 1,…,n Costs O(n) time on unit cost RAM Avoids (n log n) lower bound on SORT by avoiding comparisons instead uses indexing of RAM Generalizes (in c passes) to key domains {1, …, n c }

Procedure RADIXSORT Input 1)for j = 1, …, n do initialize B[j] to be the empty list 2)for I=1, …, n do add I to B[x i ] 3)Let L = (i 1, i 2, …, i n ) be the concatenation of B[1],…,B[n] 4)output

Deterministic Selection and Sorting Prepared by John Reif, Ph.D. Analysis of Algorithms