Chapter 9: Selection of Order Statistics What are an order statistic? min, max median, i th smallest, etc. Selection means finding a particular order statistic Selection by sorting T(n) =  (nlgn) Selection in linear time best case worst case average case

Given a set of n elements, i th order statistic = i th smallest element min is 1 st order statistic; max is the n th order statistic parity of a set is whether n is even or odd median is roughly half way between min and max unique for an odd parity set i th smallest with i = (n+1)/2 regardless of parity lower median means i th smallest with i =  (n+1)/2  upper median means i th smallest with i =  (n+1)/2  Min, Max and Median order statistics

Find the i th order statistic in set of n (distinct) elements A= (i.e. find x  A that x is larger than exactly i –1 other elements of A) Selection problem can be solve in O(nlgn) by sorting Since min and max can be found in linear time, expect that any order statistic can be found in linear time. Analyze deterministic algorithm, SELECT, that finds the i th order statistic with linear a worst-case runtime. Analyze RANDOMIZED-SELECT that finds the i th order statistic by randomized partition and has an expected runtime O(n) The selection problem

Select-by-Partition(A,p,r,i) %argument i specifies which order statistic 1if p=r then return A[p] %single element is i th smallest by default 2q  Partition(A,p,r) %get upper and lower sub-arrays 3k  q – p + 1 %number of elements in lower including pivot 4if i = k then 5return A[q] %pivot is the i th smallest element 6else 7if i < k then return Select-by-Partition(A,p,q-1,i) 8else 9return Select-by-Partition(A,q+1,r,i - k) Select by partition pseudocode Note: index of i th order statistic changed in upper sub-array With favorable splits, T(n) = O(n) Why not O(nlg(n)) as in quicksort?

Selection algorithm with worst-case runtime = O(n) Possible to design a deterministic selection algorithm that has a linear worst-case runtime. Making the pivot an input parameter, can guarantee a good split when partition is called Processing before calling partition determines a good choice for pivot.

SELECT by partition with preprocessing: T(n)= O(n) Step 1: Divide n-element sequence into floor(n/5) groups of 5 elements and at most one with less than 5: cost =  (n) Step 2: Use insertion sort to find median of each subgroup: cost = constant (cost of sorting 5 elements) x number of subgroups =  (n) Step 3: Use SELECT to find the median of the medians: cost = T(ceiling(n/5)) The median of the group that may contain less than 5 is included. Step 4: Partition the input array with pivot = median of medians. Calculate k, the number of elements < pivot: cost =  (n) + constant. If k=i return pivot. Step 5: If pivot is not the i th smallest element, bound the runtime by the time by assuming the i th smallest element is in larger sub-array: cost < T(7n/10 + 6) (explained below)

Diagram to help explain cost of Step 5 Dots represent elements of input. Subgroups of 5 occupy columns Arrows point from larger to smaller elements. Medians are white. x marks median of medians. Shaded area shows elements greater than x 3 out of 5 are shaded if subgroup is full and does not contain x

Odd number in full groups so that median is unique Total number 28 so that partial group has unique median Lower median of medians so that no elements > x are in groups with median < x Rationale for this diagram

LB(A(k)>x) = 3((n/5)/2-2) = 3n/10-6 Value of constants in formula are easy to rationalize 3 for number in full groups (n/5)/2 – 2 for approximate number of full groups Conservative approximation because we know there is at least one partial group contain elements with A(k)>x Simple formula for lower bound on number of elements > x

LB(A(k)>x) = 3((n/5)/2-2) = 3n/10-6 UB(A(k)<x) = n – (3n/10-6) = 7n/10+6 By similar set of arguments, UB(A(k)>x) = 7n/10+6 7n/10+6 is a conservative estimate of the size of the larger subarray when partition is called with pivot equal to median of medians Worst case described by T(n) < T(ceiling(n/5)) + T(7n/10+6) +  (n) Simple formula for upper bound on number of elements < x

By similar arguments, 7n/10+6 is shown to be an upper bound on the number of elements of A with value less than x

Show by substitution that T(n) = T(ceiling(n/5)) + T(7n/10+6) +  (n) has asymptotic solution T(n) = O(n).

CS 350 Spring 2016 [All problems are from Cormen et al, 3 rd Edition] Homework Assignment 15: due 4/6/16 Ex p 223: (a) Show that SELECT with groups of 7 has a linear worst-case runtime (b) Show that SELECT with groups of 3 does not run in linear time.

CS 350 Spring 2016 [All problems are from Cormen et al, 3 rd Edition] Homework Assignment 16: due 4/8/16 1. ex p ex p 223 Write a pseudo code (variation of codes in text) Explain how code works Analyze its run time

Randomized-Select lets us analyze the runtime for the average case Randomized-Select(A,p,r,i) 1if p=r then return A[p] 2q  Randomized-Partition(A,p,r) 3k  q – p + 1 4if i = k then 5return A[q] (pivot is the i th smallest element) 6else 7if i < k then return Randomized-Select(A,p,q-1,i) 8else 9return Randomized-Select(A,q+1,r,i –k) As in Randomized-Quicksort, Randomized-Partition chooses a pivot at random from array elements between p and r

Upper bound on the expected value of T(n) for Randomized-Select Call to Randomized-Partition creates upper and lower sub-arrays Include the pivot in lower sub-array A(p..q) Define indicator random variables X k = I{sub-array A[p...q]} has exactly k elements} 1 < k < n All possibilities values of k are equally likely. E[X k ] = 1/n

Assume that ith smallest always falls in larger partition This assumption ensures an upper bound on E(T(n)) T(n) < {X k T(max(k-1,n-k))} + O(n) randomized recurrence T(n) = T(n-1) + O(n) when lower sub-array has 1 element T(n) = T(n-2) + O(n) when lower sub-array has 2 element. T(n) = T(n-2) + O(n) when lower sub-array has n-1 element T(n) = T(n-1) + O(n) when lower sub-array has n element

E[T(n)] < { E[X k T(max(k-1,n-k))] } + O(n) (linearity of expected values) E[T(n)] < { E[X k ] E[ T(max(k-1,n-k))] } + O(n) (expected value of independent of random variables) E[T(n)] < (1/n) E[ T(max(k-1,n-k))] + O(n) (using E[X k ] = 1/n)

E[T(n)] < (1/n) E[ T(max(k-1,n-k))] + O(n) if k >  n/2 , max(k-1,n-k) = k-1 if k <  n/2 , max(k-1,n-k) = n-k For even n, each term from T(n/2) to T(n-1) occurs exactly twice Similar argument applies for odd n E[T(n)] < (2/n) E[ T(k)] + O(n) (using the redundancy of T’s) E[T(n)] < (2/n) { E[ T(k)] - E[ T(k)] } + O(n) (Get setup to use the arithmetic sum)

Apply substitution method: assume E[T(k)] = O(k) Then exist c > 0 such that E[T(k)] < ck E[T(n)] 0 Now use arithmetic sum After much algebra (text p219) E[T(n)] < cn – (cn/4 – c/2 – dn) Find c and n 0