Presentation is loading. Please wait.

Presentation is loading. Please wait.

A simple example finding the maximum of a set S of n numbers.

Similar presentations

Presentation on theme: "A simple example finding the maximum of a set S of n numbers."— Presentation transcript:

1 A simple example finding the maximum of a set S of n numbers

2 Time complexity: Calculation of T(n): Assume n = 2 k, T(n)= 2T(n/2)+1 = 2(2T(n/4)+1)+1 = 4T(n/4)+2+1 : =2 k-1 T(2)+2 k-2 + … =2 k-1 +2 k-2 + … =2 k -1 = n-1 Time complexity

3 A general divide-and-conquer algorithm Step 1: If the problem size is small, solve this problem directly; otherwise, split the original problem into 2 sub-problems with equal sizes. Step 2: Recursively solve these 2 sub-problems by applying this algorithm. Step 3: Merge the solutions of the 2 sub- problems into a solution of the original problem.

4 Time complexity of the general algorithm Time complexity: where S(n) : time for splitting M(n) : time for merging b : a constant c : a constant e.g. Binary search e.g. quick sort e.g. merge sort

5 Divide and Conquer Divide-and-conquer method for algorithm design: – Divide: if the input size is too large to deal with in a straightforward manner, divide the problem into two or more disjoint subproblems – Conquer: use divide and conquer recursively to solve the subproblems – Combine: take the solutions to the subproblems and “merge” these solutions into a solution for the original problem

6 Recurrences Running times of algorithms with Recursive calls can be described using recurrences A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs Example: Merge Sort

7 Solving Recurrences Repeated substitution method – Expanding the recurrence by substitution and noticing patterns Substitution method – guessing the solutions – verifying the solution by the mathematical induction Recursion-trees Master method – templates for different classes of recurrences

8 Repeated Substitution Method Let’s find the running time of merge sort (let’s assume that n=2 b, for some b).

9 Repeated Substitution Method The procedure is straightforward: – Substitute – Expand – Substitute – Expand – … – Observe a pattern and write how your expression looks after the i-th substitution – Find out what the value of i (e.g., lgn) should be to get the base case of the recurrence (say T(1)) – Insert the value of T(1) and the expression of i into your expression

10 Integer Multiplication Algorithm: Multiply two n-bit integers I and J. – Divide step: Split I and J into high-order and low-order bits – We can then define I*J by multiplying the parts and adding: – So, T(n) = 4T(n/2) + n, which implies T(n) is O(n 2 ). – But that is no better than the algorithm we learned in grade school.

11 An Improved Integer Multiplication Algorithm Algorithm: Multiply two n-bit integers I and J. – Divide step: Split I and J into high-order and low-order bits – Observe that there is a different way to multiply parts: – So, T(n) = 3T(n/2) + n, which implies T(n) is O(n log 2 3 ), by the Master Theorem. – Thus, T(n) is O(n ).

12 Matrix multiplication Let A, B and C be n  n matrices C = AB C(i, j) = A(i, k)B(k, j) The straightforward method to perform a matrix multiplication requires O(n 3 ) time.

13 Divide-and-conquer approach C = AB C 11 = A 11 B 11 + A 12 B 21 C 12 = A 11 B 12 + A 12 B 22 C 21 = A 21 B 11 + A 22 B 21 C 22 = A 21 B 12 + A 22 B 22 Time complexity: (# of additions : n 2 ) We get T(n) = O(n 3 )

14 P = (A 11 + A 22 )(B 11 + B 22 ) Q = (A 21 + A 22 )B 11 R = A 11 (B 12 - B 22 ) S = A 22 (B 21 - B 11 ) T = (A 11 + A 12 )B 22 U = (A 21 - A 11 )(B 11 + B 12 ) V = (A 12 - A 22 )(B 21 + B 22 ). C 11 = P + S - T + V C 12 = R + T C 21 = Q + S C 22 = P + R - Q + U Strassen ’ s matrix multiplicaiton

15 Time complexity 7 multiplications and 18 additions or subtractions Time complexity: T(n) = an 2 + 7T(n/2) = an 2 + 7(a(n/2) 2 + 7T(n/4)) = an 2 + (7/4)an T(n/4) = … : = an 2 (1 + 7/4 + (7/4) 2 + … +(7/4) k-1 +7 k T(1))  cn 2 (7/4) log2n +7 log2n, c is a constant = cn log24+log27-log24 +n log27 = O(n log27 )  O(n 2.81 )

16 Median Finding Given a set of "n" unordered numbers we want to find the "k th " smallest number. (k is an integer between 1 and n).

17 A Simple Solution A simple sorting algorithm like heapsort will take Order of O(nlg 2 n) time. Step Running Time Sort n elements using heapsortO(nlog 2 n) Return the k th smallest element O(1) Total running time O(nlog 2 n)

18 Linear Time selection algorithm Also called Median Finding Algorithm. Find k th smallest element in O (n) time in worst case. Uses Divide and Conquer strategy. Uses elimination in order to cut down the running time substantially.

19 Steps to solve the problem Step 1: If n is small, for example n<6, just sort and return the k th smallest number in constant time i.e; O(1) time. Step 2: Group the given number in subsets of 5 in O(n) time.

20 Step3: Sort each of the group in O (n) time. Find median of each group. Given a set (……..2,5,9,19,24,54,5,87,9,10,44,32,21,13,24, 18,26,16,19,25,39,47,56,71,91,61,44,28………) having n elements.

21 ……………….. 21 Arrange the numbers in groups of five

22 Find median of N/5 groups ……………….. 21 Median of each group

23 ……………….. Find the Median of each group Find m,the median of medians 21 3.n/10

24 Find the sets L and R Compare the n-1 elements with the median m and find two sets L and R such that every element in L is smaller than m and every element in R is greater than m. m L R 3n/10

25 Description of the Algorithm step If n is small, for example n<6, just sort and return the k the smallest number.(time- 7) If n>5, then partition the numbers into groups of 5.(time n) Sort the numbers within each group. Select the middle elements (the medians). (time- 7n/5) Call your "Selection" routine recursively to find the median of n/5 medians and call it m. (time-T n/5 ) Compare all n-1 elements with the median of medians m and determine the sets L and R, where L contains all elements m. Clearly, the rank of m is r=|L|+1 (|L| is the size of L) (time- n)

26 Contd…. If k=r, then return m If kr, then return k-r th smallest of the set R.

27 Recursive formula T (n)=O(n) + T(n/5) +T(7n/10) We will solve this equation in order to get the complexity. We assume that T (n)< C*n T (n)= a*n + T (n/5) + T (7n/10) <= C*n/5+ C*7*n/10 + a*n <= C*n if 9*C/10 +a <= C Which implies C >= 10*a Hence 10*a*n satisfies the recurrence and so T(n) is O(n)

28 2-D ranking finding Def: Let A = (a 1,a 2 ), B = (b 1,b 2 ). A dominates B iff a 1 > b 1 and a 2 > b 2 Def: Given a set S of n points, the rank of a point x is the number of points dominated by x. B A C D E rank(A)= 0 rank(B) = 1 rank(C) = 1 rank(D) = 3 rank(E) = 0

29 Straightforward algorithm: compare all pairs of points : O(n 2 )

30 Input: A set S of planar points P 1,P 2,…,P n Output: The rank of every point in S Step 1: (Split the points along the median line L into A and B.) a. If S contains only one point, return its rank as 0. b. Else, choose a cut line L perpendicular to the x-axis such that n/2 points of S have x-values L (call this set of points A) and the remainder points have x-values L(call this set B). Note that L is a median x-value of this set. Step 2: Find ranks of points in A and ranks of points in B, recursively. Step 3: Sort points in A and B according to their y-values. Scan these points sequentially and determine, for each point in B, the number of points in A whose y-values are less than its y-value. The rank of this point is equal to the rank of this point among points in B, plus the number of points in A whose y-values are less than its y-value. Divide-and-conquer 2-D ranking finding


32 2-D maxima finding problem Def : A point (x 1, y 1 ) dominates (x 2, y 2 ) if x 1 > x 2 and y 1 > y 2. A point is called a maxima if no other point dominates it Straightforward method : Compare every pair of points. Time complexity: O(n 2 )

33 Divide-and-conquer for maxima finding The maximal points of S L and S R

34 The algorithm: Input: A set of n planar points. Output: The maximal points of S. Step 1: If S contains only one point, return it as the maxima. Otherwise, find a line L perpendicular to the X-axis which separates the set of points into two subsets S L and S R, each of which consisting of n/2 points. Step 2: Recursively find the maximal points of S L and S R. Step 3: Find the largest y-value of S R. Project the maximal points of S L onto L. Discard each of the maximal points of S L if its y-value is less than the largest y-value of S R.

35 Time complexity: T(n) Step 1: O(n) Step 2: 2T(n/2) Step 3: O(n) Assume n = 2 k T(n) = O(n log n)

36 The closest pair problem Given a set S of n points, find a pair of points which are closest together. 1-D version : Solved by sorting Time complexity : O(n log n)  2-D version

37 at most 6 points in area A:

38 The algorithm: Input: A set of n planar points. Output: The distance between two closest points. Step 1: Sort points in S according to their y-values and x-values. Step 2: If S contains only two points, return infinity as their distance. Step 3: Find a median line L perpendicular to the X- axis to divide S into two subsets, with equal sizes, S L and S R. Step 4: Recursively apply Step 2 and Step 3 to solve the closest pair problems of S L and S R. Let d L (d R ) denote the distance between the closest pair in S L (S R ). Let d = min(d L, d R ).

39 Step 5: For a point P in the half-slab bounded by L-d and L, let its y-value by denoted as y P. For each such P, find all points in the half-slab bounded by L and L+d whose y-value fall within y P +d and y P - d. If the distance d between P and a point in the other half-slab is less than d, let d=d. The final value of d is the answer. Time complexity: O(n log n) Step 1: O(n log n) Steps 2~5:  T(n) = O(n log n)

40 The convex hull problem The convex hull of a set of planar points is the smallest convex polygon containing all of the points. concave polygon:convex polygon:

41 The divide-and-conquer strategy to solve the problem:

42 The merging procedure: 1.Select an interior point p. 2.There are 3 sequences of points which have increasing polar angles with respect to p. (1) g, h, i, j, k (2) a, b, c, d (3) f, e 3.Merge these 3 sequences into 1 sequence: g, h, a, b, f, c, e, d, i, j, k. 4.Apply Graham scan to examine the points one by one and eliminate the points which cause reflexive angles.

43 e.g. points b and f need to be deleted. Final result:

44 Divide-and-conquer for convex hull Input : A set S of planar points Output : A convex hull for S Step 1: If S contains no more than five points, use exhaustive searching to find the convex hull and return. Step 2: Find a median line perpendicular to the X-axis which divides S into S L and S R ; S L lies to the left of S R. Step 3: Recursively construct convex hulls for S L and S R. Denote these convex hulls by Hull(S L ) and Hull(S R ) respectively.

45 Step 4: Apply the merging procedure to merge Hull(S L ) and Hull(S R ) together to form a convex hull. Time complexity: T(n) = 2T(n/2) + O(n) = O(n log n)

46 The Fast Fourier Transform

47 Outline Polynomial Multiplication Problem Primitive Roots of Unity The Discrete Fourier Transform The FFT Algorithm

48 Polynomials Polynomial: In general,

49 Polynomial Evaluation Horner’s Rule: – Given coefficients (a 0,a 1,a 2,…,a n-1 ), defining polynomial – Given x, we can evaluate p(x) in O(n) time using the equation Eval(A,x): [Where A=(a 0,a 1,a 2,…,a n-1 )] – If n=1, then return a 0 – Else, Let A’=(a 1,a 2,…,a n-1 ) [assume this can be done in constant time] return a 0 +x*Eval(A’,x)

50 Polynomial Multiplication Given coefficients (a 0,a 1,a 2,…,a n-1 ) and (b 0,b 1,b 2,…,b n-1 ) defining two polynomials, p() and q(), and number x, compute p(x)q(x). Horner’s rule doesn’t help, since where A straightforward evaluation would take O(n 2 ) time. The “magical” FFT will do it in O(n log n) time.

51 Polynomial Interpolation & Polynomial Multiplication Given a set of n points in the plane with distinct x-coordinates, there is exactly one (n-1)-degree polynomial going through all these points. Alternate approach to computing p(x)q(x): – Calculate p() on 2n x-values, x 0,x 1,…,x 2n-1. – Calculate q() on the same 2n x values. – Find the (2n-1)-degree polynomial that goes through the points {(x 0,p(x 0 )q(x 0 )), (x 1,p(x 1 )q(x 1 )), …, (x 2n-1,p(x 2n-1 )q(x 2n-1 ))}. Unfortunately, a straightforward evaluation would still take O(n 2 ) time, as we would need to apply an O(n)-time Horner’s Rule evaluation to 2n different points. The “magical” FFT will do it in O(n log n) time, by picking 2n points that are easy to evaluate…


53 Primitive Roots of Unity A number  is a primitive n-th root of unity, for n>1, if –  n = 1 – The numbers 1, ,  2, …,  n-1 are all distinct Example 1: – Z * 11 : – 2, 6, 7, 8 are 10-th roots of unity in Z * 11 – 2 2 =4, 6 2 =3, 7 2 =5, 8 2 =9 are 5-th roots of unity in Z * 11 – 2 -1 =6, 3 -1 =4, 4 -1 =3, 5 -1 =9, 6 -1 =2, 7 -1 =8, 8 -1 =7, 9 -1 =5 Example 2: The complex number e 2  i/n is a primitive n-th root of unity, where




57 Properties of Primitive Roots of Unity Inverse Property: If  is a primitive root of unity, then  -1 =  n-1 – Proof:  n-1 =  n =1 Cancellation Property: For non-zero -n

58 The Discrete Fourier Transform Given coefficients (a 0,a 1,a 2,…,a n-1 ) for an (n-1)-degree polynomial p(x) The Discrete Fourier Transform is to evaluate p at the values – 1, ,  2,…,  n-1 – We produce (y 0,y 1,y 2,…,y n-1 ), where y j =p(  j ) – That is, – Matrix form: y=Fa, where F[i,j]=  ij. The Inverse Discrete Fourier Transform recovers the coefficients of an (n-1)-degree polynomial given its values at 1, ,  2,…,  n-1 – Matrix form: a=F -1 y, where F -1 [i,j]=  -ij /n.

59 Correctness of the inverse DFT The DFT and inverse DFT really are inverse operations Proof: Let A=F -1 F. We want to show that A=I, where If i=j, then If i and j are different, then

60 Convolution The DFT and the inverse DFT can be used to multiply two polynomials So we can get the coefficients of the product polynomial quickly if we can compute the DFT (and its inverse) quickly…

61 The Fast Fourier Transform The FFT is an efficient algorithm for computing the DFT The FFT is based on the divide-and-conquer paradigm: – If n is even, we can divide a polynomial into two polynomials and we can write

62 The FFT Algorithm The running time is O(n log n). [inverse FFT is similar]

Download ppt "A simple example finding the maximum of a set S of n numbers."

Similar presentations

Ads by Google