CSE 421 Algorithms Richard Anderson Lecture 13 Divide and Conquer

What you really need to know about recurrences Work per level changes geometrically with the level Geometrically increasing (x > 1) –The bottom level wins Geometrically decreasing (x < 1) –The top level wins Balanced (x = 1) –Equal contribution

T(n) = aT(n/b) + n c Balanced: a = b c Increasing: a > b c Decreasing: a < b c

Classify the following recurrences (Increasing, Decreasing, Balanced) T(n) = n + 5T(n/8) T(n) = n + 9T(n/8) T(n) = n 2 + 4T(n/2) T(n) = n 3 + 7T(n/2) T(n) = n 1/2 + 3T(n/4)

Divide and Conquer Algorithms Split into sub problems Recursively solve the problem Combine solutions Make progress in the split and combine stages –Quicksort – progress made at the split step –Mergesort – progress made at the combine step

Closest Pair Problem Given a set of points find the pair of points p, q that minimizes dist(p, q)

Divide and conquer If we solve the problem on two subsets, does it help? (Separate by median x coordinate) 11 22

Packing Lemma Suppose that the minimum distance between points is at least , what is the maximum number of points that can be packed in a ball of radius  ?

Combining Solutions Suppose the minimum separation from the sub problems is  In looking for cross set closest pairs, we only need to consider points with  of the boundary How many cross border interactions do we need to test?

A packing lemma bounds the number of distances to check 

Details Preprocessing: sort points by y Merge step –Select points in boundary zone –For each point in the boundary Find highest point on the other side that is at most  above Find lowest point on the other side that is at most  below Compare with the points in this interval (there are at most 6)

Identify the pairs of points that are compared in the merge step following the recursive calls

Algorithm run time After preprocessing: –T(n) = cn + 2 T(n/2)

Divide and Conquer Algorithms Mergesort, Quicksort Strassen’s Algorithm Closest Pair Algorithm (2d) Inversion counting Integer Multiplication (Karatsuba’s Algorithm) FFT –Polynomial Multiplication –Convolution

Inversion Problem Let a 1,... a n be a permutation of 1.. n (a i, a j ) is an inversion if i a j Problem: given a permutation, count the number of inversions This can be done easily in O(n 2 ) time –Can we do better? 4, 6, 1, 7, 3, 2, 5

Counting Inversions Count inversions on lower half Count inversions on upper half Count the inversions between the halves

Count the Inversions

Problem – how do we count inversions between sub problems in O(n) time? Solution – Count inversions while merging Standard merge algorithm – add to inversion count when an element is moved from the upper array to the solution

Use the merge algorithm to count inversions Indicate the number of inversions for each element detected when merging

Inversions Counting inversions between two sorted lists –O(1) per element to count inversions Algorithm summary –Satisfies the “Standard recurrence” –T(n) = 2 T(n/2) + cn x x x x x x x x y y y y y y y y z z z z z z z z z z z z z z z z

Integer Arithmetic X Runtime for standard algorithm to add two n digit numbers: Runtime for standard algorithm to multiply two n digit numbers:

Recursive Algorithm (First attempt) x = x 1 2 n/2 + x 0 y = y 1 2 n/2 + y 0 xy = (x 1 2 n/2 + x 0 ) (y 1 2 n/2 + y 0 ) = x 1 y 1 2 n + (x 1 y 0 + x 0 y 1 )2 n/2 + x 0 y 0 Recurrence: Run time:

Simple algebra x = x 1 2 n/2 + x 0 y = y 1 2 n/2 + y 0 xy = x 1 y 1 2 n + (x 1 y 0 + x 0 y 1 ) 2 n/2 + x 0 y 0 p = (x 1 + x 0 )(y 1 + y 0 ) = x 1 y 1 + x 1 y 0 + x 0 y 1 + x 0 y 0

Karatsuba’s Algorithm Multiply n-digit integers x and y Let x = x 1 2 n/2 + x 0 and y = y 1 2 n/2 + y 0 Recursively compute a = x 1 y 1 b = x 0 y 0 p = (x 1 + x 0 )(y 1 + y 0 ) Return a2 n + (p – a – b)2 n/2 + b Recurrence: T(n) = 3T(n/2) + cn

FFT, Convolution and Polynomial Multiplication Preview –FFT - O(n log n) algorithm Evaluate a polynomial of degree n at n points in O(n log n) time –Computation of Convolution and Polynomial Multiplication (in O(n log n)) time

Complex Analysis Polar coordinates: re  i e  i = cos  + i sin  a is a n th root of unity if a n = 1 Square roots of unity: +1, -1 Fourth roots of unity: +1, -1, i, -i Eighth roots of unity: +1, -1, i, -i,  + i ,  - i , -  + i , -  - i  where  = sqrt(2)

e 2  ki/n e 2  i = 1 e  i = -1 n th roots of unity: e 2  ki/n for k = 0 …n-1 Notation:  k,n = e 2  ki/n Interesting fact: 1 +  k,n +  2 k,n +  3 k,n  n-1 k,n = 0 for k != 0

Convolution a 0, a 1, a 2,..., a m-1 b 0, b 1, b 2,..., b n-1 c 0, c 1, c 2,...,c m+n-2 where c k =  i+j=k a i b j

Applications of Convolution Polynomial Multiplication Signal processing –Gaussian smoothing –Sequence a 1, a 2,..., a n –Mask, w -k, w -(k-1),..., w -1, w 0, w 1,..., w k-1, w k Addition of random variables

FFT Overview Polynomial interpolation –Given n+1 points (x i,y i ), there is a unique polynomial P of degree at most n which satisfies P(x i ) = y i

Polynomial Multiplication n-1 degree polynomials A(x) = a 0 + a 1 x + a 2 x 2 + … +a n-1 x n-1, B(x) = b 0 + b 1 x + b 2 x 2 + …+ b n-1 x n-1 C(x) = A(x)B(x) C(x)=c 0 +c 1 x + c 2 x 2 + … + c 2n-2 x 2n-2 p 1, p 2,..., p 2n A(p 1 ), A(p 2 ),..., A(p 2n ) B(p 1 ), B(p 2 ),..., B(p 2n ) C(p i ) = A(p i )B(p i ) C(p 1 ), C(p 2 ),..., C(p 2n )

FFT Polynomial A(x) = a 0 + a 1 x a n-1 x n-1 Compute A(  j,n ) for j = 0,..., n-1 For simplicity, n is a power of 2

Useful trick A(x) = a 0 + a 1 x + a 2 x 2 + a 3 x a n-1 x n-1 A even (x) = a 0 + a 2 x + a 4 x a n-2 x (n-2)/2 A odd (x) = a 1 + a 3 x + a 5 x 2 + …+ a n-1 x (n-2)/2 Show: A(x) = A even (x 2 ) + x A odd (x 2 )

Lemma:  2 j,2n =  j,n Squares of 2n th roots of unity are n th roots of unity

FFT Algorithm // Evaluate the 2n-1 th degree polynomial A at //  0,2n,  1,2n,  2,2n,...,  2n-1,2n FFT(A, 2n) Recursively compute FFT(A even, n) Recursively compute FFT(A odd, n) for j = 0 to 2n-1 A(  j,2n ) = A even (  2 j,2n ) +  j,2n A odd (  2 j,2n )

Polynomial Multiplication n-1 th degree polynomials A and B Evaluate A and B at  0,2n,  1,2n,...,  2n-1,2n Compute C(  j,2n ) for j = 0 to 2n -1 We know the value of a 2n-2 th degree polynomial at 2n points – this determines a unique polynomial, we just need to determine the coefficients

Now the magic happens... C(x) = c 0 + c 1 x + c 2 x 2 + … + c 2n-1 x 2n-1 (we want to compute the c i ’s) Let d j = C(  j,2n ) D(x) = d 0 + d 1 x + d 2 x 2 + … + d 2n-1 x 2n-1 Evaluate D(x) at the 2n th roots of unity D(  j,2n ) = [see text for details] = 2nc 2n-j

Polynomial Interpolation Build polynomial from the values of C at the 2n th roots of unity Evaluate this polynomial at the 2n th roots of unity