 한양대학교 정보보호 및 알고리즘 연구실 이재준 담당교수님 : 박희진 교수님

Presentation on theme: "한양대학교 정보보호 및 알고리즘 연구실 이재준 담당교수님 : 박희진 교수님"— Presentation transcript:

한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님
Recurrences 한양대학교 정보보호 및 알고리즘 연구실 이재준 담당교수님 : 박희진 교수님

Contents of Table A. Growth of function B. Recurrence
Relationship between asymptotic notations B. Recurrence Recurrence solution methods The substitution method The recursion-tree method

Analogy f(n) = Θ(g(n)) ≈ f(n) = g(n). f(n) = O(g(n)) ≈ f(n) ≤ g(n).
3. Relationship between this notations Analogy f(n) = Θ(g(n)) ≈ f(n) = g(n). f(n) = O(g(n)) ≈ f(n) ≤ g(n). f(n) = Ω(g(n)) ≈ f(n) ≥ g(n). f(n) = o(g(n)) ≈ f(n) < g(n). f(n) = ω(g(n)) ≈ f(n) > g(n).

Transitivity Transpose symmetry 3. Relationship between this notations
f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) , f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) , f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) , f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)). f(n) = O(g(n)) if and only if g(n) = Ω(f(n)), f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

Reflexivity Symmetry f(n) = Θ(f(n)) f(n) = O(f(n)) f(n) = Ω(f(n))
3. Relationship between this notations Reflexivity f(n) = Θ(f(n)) f(n) = O(f(n)) f(n) = Ω(f(n)) Symmetry f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).

When should we use the recurrence?
Recurrences Recurrence When should we use the recurrence? When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.

Recurrences Recurrence - Definition
Equation or inequality that describes a function in terms of its value on smaller inputs.

Recurrences Recurrence Example if n=1
Worst-case running time of T(n) could be described by the recurrence Merge-Sort procedure Show that the solution of T(n) = Θ(n log n) if n=1 if n>1

Recurrences Recurrence Solution method to example
1) The substitution method guess a bound and prove our guess correct. 2) The recursion-tree method convert the recurrence into a tree whose nodes represent the costs incurred at various level of recursion 3) The master method determining asymptotic bounds for many simple recurrences of the form 범위에 대한 가정을 하고 그 가정이 맞는지 수학적 귀납법을 이용하여 증명하는것입니다. Recurrence를 트리형태로 바꾸고 각 트리레벨에서의 cost를 구한후 이들의 총합을 구함으로써 전체 cost를 구하는 방법입니다. Recurrence 의 기본적인 Form에서 주어진 세가지 경우에 따라 값을 구하는 방법입니다. ( a ≥ 1, b ≥ 1, and f(n) is given function )

1. Technicalities for abbreviation
1) assumption of integer The running time T(n) of algorithm is only defined when n is an integer 2) floors / ceilings We often omit floors and ceilings 3) boundary condition The running time of an algorithm on a constant-sized input is a constant that we typically ignore.

1. Technicalities for abbreviation
Technicalities Example The recurrence describing the worst-case running time of Merge-Sort if n=1 if n>1 assumption of integer Omit floors and ceilings 생략되는 Technical details은 recurrence solution method를 보면서 보게 될 것이다. Floor : 같거나 더 큰 수치 중에서 가장 작은 정수 Ceiling : 같거나 더 작은 수치 중에서 가장 큰 정수 Omit boundary condition

2. Recurrence solution methods
Substitution Method - Definition 1. Guess the form of the solution 2. Use mathematical induction to find the constant and show that the solution works. - It can be used to establish either upper or lower bounds on a recurrence.

2. Recurrence solution methods
Substitution Method Example Let us determine an upper bound on the recurrence

2. Recurrence solution methods
Substitution Method Solution to example 1. guess that the solution is 2. prove that ( c > 0 ) We need to find constant c and n0 to meet the requirement about assumption T(n) ≤ cn lg n ( c>0 ) We using the Mathematical induction to show that our assumption holds for the boundary condition.

2. Recurrence solution methods
Substitution Method Solution to example Mathematical induction We can prove something stronger for a given value by assuming something stronger for smaller values. → T(1), T(2), T(3), … ….. T(⌊n/2⌋ ), T(n) k k+1 k means ⌊n/2⌋ k+1 means n

2. Recurrence solution methods
Substitution Method Solution to example Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn k means k+1 means n -> Substitution T (⌊n/2⌋) ≤ c ⌊n/2⌋ lg (⌊n/2⌋) T(n) ≤ 2(c ⌊n/2⌋lg(⌊n/2⌋)) + n ≤ cn lg(n/2) + n ( because, ⌊n/2⌋ < n/2 ) = cn lg n - cn lg 2 + n = cn lg n -(c+1)n ( It’s our assumed fact )

2. Recurrence solution methods
Substitution Method Solution to example Assumed Fact : cn lg n –(c+1)n Prove : T(n) ≤ cn Assumed Fact ≤ Prove T(n) ≤ cn lg n – (c+1)n ≤ cn lg n (as long as c ≥ 1)

2. Recurrence solution methods
Substitution Method Solution to example - Find the constant n0 - we can take advantage of asymptotic notation, we can replace base case T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0 - So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2) Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

2. Recurrence solution methods
Substitution Method Solution to example - Find the constant c - choosing c large enough T (2) = 4 and T (3) = 5. T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ). Any choice of c ≥ 2 suffices for base case of n=2 and n =3

2. Recurrence solution methods
Substitution Method Skill - Subtleties We can prove something stronger for given value by assuming something stronger for a smaller values. Overcome difficulty by subtracting a lower-order term from our previous guess

2. Recurrence solution methods
Substitution Method Skill - Subtleties example Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn Substitution k means k+1 means n

2. Recurrence solution methods
Substitution Method Skill - Subtleties example ( It’s Impossible. Then T(n)= O(n) is wrong? ) Subtracting a lower-order term from our previous guess T(n) ≤ cn So, we now prove T(n) ≤ cn-b

2. Recurrence solution methods
Substitution Method Skill - Subtleties example Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn-b ( b≥1, c must be chosen large enough ) T(n) 기존의 증명할 부등식에 상수 b를 subtracting 마친 부등식에 대하여 증명을 하는 것입니다. Substitution을 마친 가정된 사실보다 증명하고자 하는 Big Oh의 anonymous function 이 더 클경우는 이 추측은 증명이 되는것입니다. 때문에, 상수 b가 1보다 크거나 같고, 상수 c는 충분히 크다는 조건하에 추측한 가정사항은 만족함을 증명할수 있습니다.

2. Recurrence solution methods
Substitution Method Skill - Avoid pitfalls We need to prove the exact form of the inductive hypothesis T(n) ≤ cn T(n) ≤ (c+1)n

2. Recurrence solution methods
Substitution Method Skill - Avoid pitfalls example Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn ( Wrong !!! exact form :T(n) ≤ cn)

2. Recurrence solution methods
Substitution Method Skill - Simplify this recurrence by changing variable Renaming m = lg n S(m) = T(2m) This new recurrence has same solution : S(m) = O(m log m)

2. Recurrence solution methods
Substitution Method Skill - Simplify this recurrence by changing variable This new recurrence has same solution : S(m) = O(m log m) Changing back from S(m) to T(n) T(n) = T(2m) = S(m) = O(m lg m) = O( lg n lg(lg n) )

2. Recurrence solution methods
Substitution Method Skill - Making a good guess prove loose upper and lower bounds on the recurrence and then reduce the range of uncertainty. - Guessing a solution takes : experience, occasionally, creativity We can make a good guess by using recursion tree !!

2. Recurrence solution methods
Recursion-tree - Definition For come up with good guess, convert the recurrence into a tree that each node represents the cost of a single subproblem somewhere in the set of recursive function invocations. We sum all the per-level costs to determine the total cost of all levels of the recursion Recusion-tree는 좋은 추측을 하기 위햐여 사용하는데, Recurrence를 tree로 변형을 하는데, 이 tree는, recursive function 이 발생하는 부분의 subproblem에 대한 비용으로 node를 표현하는 tree입니다.

2. Recurrence solution methods
Recursion-tree Example The recurrence is T (n) = 3T (⌊n/4⌋) + Θ(n2) = 3T (n/4) + cn2 Use the recursion tree to making a good guess for upper bound for the solution.

2. Recurrence solution methods
Recursion-tree Solution to example Draw recursion tree 3T (n/4) + cn2 cn2 T(n)

2. Recurrence solution methods
Recursion-tree Solution to example cn2 c(n/4)2 T(1) log4n When depth i then subprogram size is n/4i = 1 so, i = log4 n and it means tree’s height and tree has log4 n + 1 levels. The number of nodes at depth i is 3i and finally

2. Recurrence solution methods
Recursion-tree Solution to example Total Cost Sum of cost for root to i-1 depth Sum of cost for i depth

2. Recurrence solution methods
Recursion-tree Solution to example Total Cost is… ( When 0<x<1 then, )

2. Recurrence solution methods
Recursion-tree Solution to example Prove T (n) = O(n2) by the substitution method Recurrence : Guess : T (n) = O(n2) Prove : T (n) ≤ dn2 (for some d > 0 and for the same c > 0) T (n) = 3T (⌊n/4⌋) + Θ(n2) = 3T(⌊n/4⌋) + cn2

2. Recurrence solution methods
Recursion-tree Solution to example Find constant d T (n) = 3T(⌊n/4⌋) + cn2 T(n) ≤ 3T(⌊n/4⌋) + cn2 ≤ 3d⌊n/4⌋2 + cn2 ≤ 3d(n/4)2 + cn2 = 3/16 dn2 + cn2 ≤ dn2 (where the last step holds as long as d ≥ (16/13)c )

2. Recurrence solution methods
Recursion-tree Example The recurrence T (n) = 3T (n/3)+T(2n/3)+O(n) Use the recursion tree to making a good guess for upper bound for the solution.

2. Recurrence solution methods
Recursion-tree Solution to example Draw recursion tree T (n) = 3T (n/3)+T(2n/3)+ cn cn log3/2n Total : O(n log n) This recursion tree is not a complete binary tree. Not all leaves contribute a cost of exactly cn

2. Recurrence solution methods
Recursion-tree Solution to example Draw recursion tree T (n) = 3T (n/3)+T(2n/3)+ cn cn log3/2n The longest path from root to a leaf is n→(2/3)n → (2/3)2n → … → 1. Since, (2/3)kn = 1 when k=log3/2 n and it means tree’s height is log3/2 n . The total cost = Cost of each level x Height = cn x log3/2 n = O(cnlog3/2n) = O(n lg n)

2. Recurrence solution methods
Recursion-tree Solution to example Prove T (n) = O(n lg n) by the substitution method Recurrence : Guess : T (n) = O(n lg n) Prove : T (n) ≤ d n lg n (for some d > 0 and for the same c > 0) T(n/3) + T(2n/3) + cn

2. Recurrence solution methods
Recursion-tree Solution to example Find constant d T(n)=T(n/3) + T(2n/3) + cn T(n) ≤ T(n/3) + T(2n/3) + cn ≤ d(n/3)lg(n/3) + d(2n/3)lg(2n/3) + cn = (d(n/3)lgn - d(n/3)lg 3) + (d(2n/3)lgn – d(2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn - d((n/3)lg3 + (2n/3)lg3 - (2n/3)lg2) + cn = dnlgn - dn(lg3 - 2/3) + cn ≤ dnlgn (as long as d ≥ c/(lg 3 - (2/3)) )

Content of next week Ch4. Recurrence
Recurrence solution methods The substitution method The recursion-tree method The master method

Download ppt "한양대학교 정보보호 및 알고리즘 연구실 이재준 담당교수님 : 박희진 교수님"

Similar presentations