한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님 Chapter 3. Growth of function Chapter 4. Recurrences 한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

Contents of Table A. Growth of function B. Recurrence 2. Notation of asymptotically larger/smaller o-notation ω-notation 3. Relationship between this notations B. Recurrence 1. Technicalities for abbreviation 2. Recurrence solution methods The substitution method

f(n) is asymptotically smaller than g(n) 2. Asymptotically larger/smaller notation o-notation Asymptotically smaller - Definition for all n, n ≥ n0 , any positive constant c, c>0, 0 ≤ f(n)< cg(n), the function f(n) is smaller to g(n) to within constant factor. f(n) is asymptotically smaller than g(n) f(n) o(g(n)) n0

o-notation 2. Asymptotically larger/smaller notation - Definition o(g(n)) = { f(n) : for any positive constant c>0, there exist a constant n0 >0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0 } this limit shows the function f(n) becomes insignificant relative to g(n) as n approaches infinity

o-notation Example 1. Asymptotic notation Use the definition of o-notation to prove the following property. f(n) = o(n2)

o-notation Solution to example 2. Notation of asymptotically larger/smaller o-notation Solution to example 2n = o(n2) If f(n) = o((g(n)) then

f(n) is asymptotically larger than g(n) 2. Notation of asymptotically larger/smaller ω -notation Asymptotically larger - Definition for all n, n ≥ n0 , any positive constant c, c>0, 0 ≤ cg(n) < f(n), the function f(n) is larger to g(n) to within constant factor. f(n) is asymptotically larger than g(n) f(n) ω(g(n))

ω-notation 2. Notation of asymptotically larger/smaller - Definition ω(g(n)) = { f(n) : for any positive constant c>0, there exist a constant n0 >0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0 } if the limit exists. That is, f(n) becomes arbitrarily large relative to g(n) as n approaches infinity.

ω -notation 2. Notation of asymptotically larger/smaller - We use ω-notation to denote a lower bound that is not asymptotically tight. - We use o-notation to denote an upper bound that is not asymptotically tight. - f(n) ∈ ω(g(n)) if and only if g(n) ∈ o(f(n)). Arbitrarily : 독단적으로, 제멋대로, 마음대로

ω -notation Example 2. Notation of asymptotically larger/smaller Use the definition of o-notation to prove the following property. f(n) = ω(n)

ω -notation Solution to example 2. Notation of asymptotically larger/smaller ω -notation Solution to example =ω(n) If f(n) = ω((g(n)) then

3. Relationship between this notations f(n) O(g(n)) o(g(n)) ω(g(n)) Ω(g(n)) θ(g(n)) O(g(n)) Ω(g(n)) ω(g(n)) o(g(n))

Transitivity Transpose symmetry 3. Relationship between this notations f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) , f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) , f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) , f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)). f(n) O(g(n)) o(g(n)) ω(g(n)) Ω(g(n)) f(n) = O(g(n)) if and only if g(n) = Ω(f(n)), f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

Reflexivity Symmetry f(n) = Θ(f(n)) f(n) = O(f(n)) f(n) = Ω(f(n)) 3. Relationship between this notations Reflexivity f(n) = Θ(f(n)) f(n) = O(f(n)) f(n) = Ω(f(n)) Symmetry f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)). θ(g(n)) O(g(n)) Ω(g(n))

When should we use the recurrence? Recurrences Recurrence When should we use the recurrence? When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.

Recurrences Recurrence - Definition Equation or inequality that describes a function in terms of its value on smaller inputs.

Recurrences Recurrence Example if n=1 if n>1 Worst-case running time of T(n) could be described by the recurrence Merge-Sort procedure Show that the solution of T(n) = Θ(n log n) if n=1 if n>1 범위에 대한 가정을 하고 그 가정이 맞는지 수학적 귀납법을 이용하여 증명하는것입니다. Recurrence를 트리형태로 바꾸고 각 트리레벨에서의 cost를 구한후 이들의 총합을 구함으로써 전체 cost를 구하는 방법입니다. Recurrence 의 기본적인 Form에서 주어진 세가지 경우에 따라 값을 구하는 방법입니다.

Recurrences Recurrence Solution method to example - The substitution method guess a bound and prove our guess correct for small input. - The recursion-tree method convert the recurrence into a tree whose nodes represent the costs incurred at various level of recursion - The master method determining asymptotic bounds for many simple recurrences of the form 범위에 대한 가정을 하고 그 가정이 맞는지 수학적 귀납법을 이용하여 증명하는것입니다. Recurrence를 트리형태로 바꾸고 각 트리레벨에서의 cost를 구한후 이들의 총합을 구함으로써 전체 cost를 구하는 방법입니다. Recurrence 의 기본적인 Form에서 주어진 세가지 경우에 따라 값을 구하는 방법입니다. ( a ≥ 1, b ≥ 1, and f(n) is given function )

1. Technicalities for abbreviation - assumption of integer The running time T(n) of algorithm is only defined when n is an integer - floors / ceilings We often omit floors and ceilings - boundary condition The running time of an algorithm on a constant-sized input is a constant that we typically ignore.

1. Technicalities for abbreviation Technicalities Example The recurrence describing the worst-case running time of Merge-Sort if n=1 if n>1 Omit assumption of integer Omit floors and ceilings 생략되는 Technical details은 recurrence solution method를 보면서 보게 될 것이다. Floor : 같거나 더 큰 수치 중에서 가장 작은 정수 Ceiling : 같거나 더 작은 수치 중에서 가장 큰 정수 Omit boundary condition

2. Recurrence solution methods Substitution Method - Definition When recurrence is easy to guess form of the answer with the inductive hypothesis is applied to smaller values, substitution of the guessed answer for the function . 생략되는 Technical details은 recurrence solution method를 보면서 보게 될것이다. Floors와 ceilings의 경우 몇몇 특수한 경우를 제외하고는 걱정할 필요가 없습니다. - It can be used to establish either upper or lower bounds on a recurrence.

2. Recurrence solution methods Substitution Method Steps 1. Guess the form of the solution 2. Use mathematical induction to find the constant and show that the solution works.

2. Recurrence solution methods Substitution Method - Making a good guess prove loose upper and lower bounds on the recurrence and then reduce the range of uncertainty. - Guessing a solution takes : experience, occasionally, creativity - we can use recursion tree

2. Recurrence solution methods Substitution Method - Subtleties we can prove something stronger for given value by assuming something stronger for a smaller values. We guess that the solution is O(n), and show that T(n) ≤ cn Overcome difficulty by subtracting a lower-order term from our previous guess ( b≥1, c must be chosen large enough )

2. Recurrence solution methods Substitution Method - Avoid pitfalls In recurrence we can falsely prove T(n)= O(n) by guessing T(n) ≤ cn

2. Recurrence solution methods Substitution Method Example Let us determine an upper bound on the recurrence

2. Recurrence solution methods Substitution Method Solution to example 1. guess that the solution is 2. prove that ( c > 0 ) We need to find constant c and n0 - Assume that this bound holds for ⌊n/2⌋, that is, T (⌊n/2⌋) ≤ c ⌊n/2⌋ lg(⌊n/2⌋). T(n) ≤ 2(c ⌊n/2⌋lg(⌊n/2⌋)) + n ≤ cn lg(n/2) + n ( because, ⌊n/2⌋ < n/2 ) = cn lg n - cn lg 2 + n = cn lg n - cn + n ≤ cn lg n (as long as c ≥ 1)

2. Recurrence solution methods Substitution Method Solution to example - Find the constant n0 - we can take advantage of asymptotic notation, we can replace base case T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0 - So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2) Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

2. Recurrence solution methods Substitution Method Solution to example - Find the constant c - choosing c large enough T (2) = 4 and T (3) = 5. T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ). Any choice of c ≥ 2 suffices for base case of n=2 and n =3

2. Recurrence solution methods Substitution Method Example Let us determine an upper bound on the recurrence

2. Recurrence solution methods Substitution Method Solution to example - Simplify this recurrence by changing variable Renaming m = lg n S(m) = T(2m) This new recurrence has same solution : S(m) = O(m log m)

2. Recurrence solution methods Substitution Method Solution to example Simplify this recurrence by changing variable This new recurrence has same solution : S(m) = O(m log m) Changing back from S(m) to T(n) T(n) = T(2m) = S(m) = O(m lg m) = O( lg n lg(lg n) )