Download presentation
Presentation is loading. Please wait.
1
Chapter 2 Divide-and-Conquer algorithms
Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. Most common usage. Break up problem of size n into two equal parts of size ½n. Solve two parts recursively. Combine two solutions into overall solution in linear time. Consequence. Brute force: n2. Divide-and-conquer: n log n. 1
2
Example 1 Integer Multiplication
"Divide and conquer", Caesar's other famous quote "I came, I saw, I conquered" Divide-and-conquer idea dates back to Julius Caesar. Favorite war tactic was to divide an opposing army in two halves, and then assault one half with his entire force.
3
Integer Arithmetic Add. Given two n-digit integers a and b, compute a + b. O(n) bit operations. Multiply. Given two n-digit integers a and b, compute a × b. Brute force solution: (n2) bit operations. 1 * Multiply 1 + Add 3
4
Divide-and-Conquer Multiplication: Warmup
To multiply two n-digit integers: Multiply four ½n-digit integers. Add two ½n-digit integers, and shift to obtain result. assumes n is a power of 2 4
5
Karatsuba’s algorithm
To multiply two n-digit integers: Add two ½n digit integers. Multiply three ½n-digit integers. Add, subtract, and shift ½n-digit integers to obtain result. Theorem. [Karatsuba-Ofman, 1962] Can multiply two n-digit integers in O(n1.585) bit operations. Karatsuba: also works for multiplying two degree N univariate polynomials A B A C C 5
6
Recursion Tree . . . . . . T(n) n T(n/2) T(n/2) T(n/2) 3(n/2) T(n/4)
T(n / 2k) 3k (n / 2k) . . . . . . T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) 3 lg n (2) 6
7
Example 2: Mergesort "Divide and conquer", Caesar's other famous quote "I came, I saw, I conquered" Divide-and-conquer idea dates back to Julius Caesar. Favorite war tactic was to divide an opposing army in two halves, and then assault one half with his entire force.
8
Sorting Sorting. Given n elements, rearrange in ascending order.
Obvious sorting applications. List files in a directory. Organize an MP3 library. List names in a phone book. Display Google PageRank results. Problems become easier once sorted. Find the median. Find the closest pair. Binary search in a database. Identify statistical outliers. Find duplicates in a mailing list. Non-obvious sorting applications. Data compression. Computer graphics. Interval scheduling. Computational biology. Minimum spanning tree. Supply chain management. Simulate a system of particles. Book recommendations on Amazon. Load balancing on a parallel computer 8
9
Mergesort Mergesort. Divide array into two halves.
Recursively sort each half. Merge two halves to make sorted whole. Jon von Neumann (1945) A L G O R I T H M S divide O(1) sort 2T(n/2) merge O(n) 9
10
Merging Merging. Combine two pre-sorted lists into a sorted whole.
How to merge efficiently? Linear number of comparisons. Use temporary array. Challenge for the bored. In-place merge. [Kronrud, 1969] A G L O R H I M S T A G H I using only a constant amount of extra storage 10
11
A Useful Recurrence Relation
Def. T(n) = number of comparisons to mergesort an input of size n. Mergesort recurrence. Solution. T(n) = O(n log2 n). Assorted proofs. We describe several ways to prove this recurrence. Initially we assume n is a power of 2 and replace with =. 11
12
Proof by Recursion Tree
T(n) n T(n/2) T(n/2) 2(n/2) T(n/4) T(n/4) T(n/4) T(n/4) 4(n/4) log2n . . . T(n / 2k) 2k (n / 2k) . . . T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) n/2 (2) n log2n 12
13
Proof by Telescoping Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n. Pf. For n > 1: assumes n is a power of 2 13
14
Proof by Induction Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n. Pf. (by induction on n) Base case: n = 1. Inductive hypothesis: T(n) = n log2 n. Goal: show that T(2n) = 2n log2 (2n). assumes n is a power of 2 14
15
Analysis of Mergesort Recurrence
Claim. If T(n) satisfies the following recurrence, then T(n) n lg n. Pf. (by induction on n) Base case: n = 1. Define n1 = n / 2 , n2 = n / 2. Induction step: assume true for 1, 2, ... , n–1. log2n 15
16
16
17
17
18
18
19
Randomized Selection Algorithm
Input: An array A, low, high, k Output: the k-th smallest element in A[low], A[low+1] … A[high] Assume: 1<= k <= high – low + 1 Algorithm RandomSelect(A, low, high, k): Step 1: m = partition(A, low, high); Step 2: if (m – low + 1 == k) return A[m] else if (m – low + 1 > k) return RandomSelect(A, low, m, k) else return RandomSelect(A, m+1, high, _________ ) End. 19
20
20
21
21
22
The Fast Fourier Transform
FFT 22
23
Outline and Reading Polynomial Multiplication Problem
Primitive Roots of Unity (page 63) The FFT Algorithm (page 64) Integer Multiplication ([AHU] Section 8.1) Java FFT Integer Multiplication FFT 23
24
Polynomials Polynomial: In general, FFT 24
25
Polynomial Evaluation
Horner’s Rule: Given coefficients (a0,a1,a2,…,an-1), defining polynomial Given x, we can evaluate p(x) in O(n) time using the equation Eval(A,x): [Where A=(a0,a1,a2,…,an-1)] If n=1, then return a0 Else, Let A’=(a1,a2,…,an-1) return a0+x*Eval(A’,x) 25
26
Polynomial Multiplication Problem
Given coefficients (a0,a1,a2,…,an-1) and (b0,b1,b2,…,bn-1) defining two polynomials, p() and q(), and number x, compute p(x)q(x). We need to compute the coefficients ci where A straightforward evaluation would take O(n2) time. FFT will do it in O(n log n) time. FFT 26
27
Polynomial Interpolation & Polynomial Multiplication
Given a set of n points in the plane with distinct x-coordinates, there is exactly one (n-1)-degree polynomial going through all these points. Alternate approach to computing p(x)q(x): Calculate p() on 2n x-values, x0,x1,…,x2n-1. Calculate q() on the same 2n x values. Find the (2n-1)-degree polynomial that goes through the points {(x0,p(x0)q(x0)), (x1,p(x1)q(x1)), …, (x2n-1,p(x2n-1)q(x2n-1))}. Unfortunately, a straightforward evaluation would still take O(n2) time, as we would need to apply an O(n)-time Horner’s Rule evaluation to 2n different points. FFT will do it in O(n log n) time, by picking 2n points that are easy to evaluate… FFT 27
28
Primitive Roots of Unity
A number w is a primitive n-th root of unity, for n > 1, if wn = 1 The numbers 1, w, w2, …, wn-1 are all distinct Example 1: Z*11: 2, 6, 7, 8 are 10-th roots of unity in Z*11 22=4, 62=3, 72=5, 82=9 are 5-th roots of unity in Z*11 2-1=6, 3-1=4, 4-1=3, 5-1=9, 6-1=2, 7-1=8, 8-1=7, 9-1=5 Example 2: The complex number e2pi/n is a primitive n-th root of unity, where 28
29
Properties of Primitive Roots of Unity
Inverse Property: If w is a primitive root of unity, then w -1=wn-1 Proof: wwn-1=wn=1 Cancellation Property: For non-zero -n<k<n, Proof: Reduction Property: If w is a primitive (2n)-th root of unity, then w2 is a primitive n-th root of unity. Proof: If 1,w,w2,…,w2n-1 are all distinct, so are 1,w2,(w2)2,…,(w2)n-1 Reflective Property: If n is even, then wn/2 = -1. Proof: By the cancellation property, for k=n/2: Corollary: wk+n/2= -wk. FFT 29
30
Fourier transform viewed as polynomial evaluation
34
The Discrete Fourier Transform
Given coefficients (a0,a1,a2,…,an-1) for an (n-1)-degree polynomial p(x) The Discrete Fourier Transform is to evaluate p at the values 1,w,w2,…,wn-1 We produce (y0,y1,y2,…,yn-1), where yj=p(wj) That is, Matrix form: y=Fa, where F[i,j]=wij. The Inverse Discrete Fourier Transform recovers the coefficients of an (n-1)-degree polynomial given its values at 1,w,w2,…,wn-1 Matrix form: a=F -1y, where F -1[i,j]=w-ij/n. FFT 34
35
Correctness of the inverse DFT
The DFT and inverse DFT really are inverse operations Proof: Let A=F -1F. We want to show that A=I, where If i=j, then If i and j are different, then FFT 35
36
Convolution The DFT and the inverse DFT can be used to multiply two polynomials So we can get the coefficients of the product polynomial quickly if we can compute the DFT (and its inverse) quickly… FFT 36
37
The Fast Fourier Transform
The FFT is an efficient algorithm for computing the DFT The FFT is based on the divide-and-conquer paradigm: If n is even, we can divide a polynomial into two polynomials and we can write FFT 37
38
The FFT Algorithm The running time is O(n log n). [inverse FFT is similar] FFT 38
39
The FFT Algorithm FFT 39
40
FFT 40
41
FFT 41
42
Using the 1D FFT for filtering
Signal = sin(7t) + .5 sin(5t) at 128 points Noise = random number bounded by .75 Filter by zeroing out FFT components < .25 CS267 Lecture 21 03/17/2011
43
Using the 2D FFT for image compression
Image = 200 x 320 matrix of values Compress by keeping largest 2.5% of FFT components Similar idea used by jpeg Jpeg breaks an image into 8x8 pixel blocks, does a DCT, quantizes the coefficients (round them so they can be Stored in very few bits), and stored these bits slide courtesy of Jim Demmel, UCB 03/17/2011
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.