1. Foundations of Algorithm 유관우
알고리즘 설계 및 분석
Foundations of Algorithm
유관우

2. Chap1. 알고리즘 설계 및 분석 (Design & Analysis of Computer Algorithms)
Motivations
“The most important course for CS students.”
Goal - “생각”
Problem solving techniques
Scientific approach
Observation, Analysis, Organization → Conclusion
Correct thinking habits
Logical writing (“The Golden Rule”)
Big know-How’s (NP-Completeness …)
Presentation techniques
“그러므로 무엇이든지 남에게 대접을 받고자 하는 대로 너희도 남들 대접하라. 이것이 율법이요 선지자니라.”
– 마태7:12
Digital Media Lab.

3. Problem : A question to which we seek an answer. Answer
Algorithm_Lecture
Problem : A question to which we seek an answer.
Answer
Solution for CS problems :
A step-by-step procedure → Algorithm
(Eg)
problem : Determine whether
instance :
answer → yes
Algorithm1 : Sequential Search
Procedure seqsearch ( n : integer; S : array[1..n] of keytype
x : keytype; var location : index);
begin
location := 1;
while location ≤ n and S[location] ≠ x do
location := location + 1;
if location > n then location := 0;
end;
Digital Media Lab.
Digital Media Lab.

4. Algorithm2 : Binary Search
Procedure binsearch ( n : integer; S : array[1..n] of keytype;
x : keytype; var location : index);
var low, high, mid : index;
begin
low = 1; high = n; location = 0;
while low ≤ high and location = 0 do
mid = ( low + high ) / 2;
if x = S[mid] then location = mid;
else if x < S[mid] then high = mid – 1;
else low = mid + 1;
end;
2 different techniques for same problem → Efficiency?
time :
space :
Digital Media Lab.

5. Fibonacci Sequence Algorithm 1 (Recursion)
function fib (n : integer) : integer;
begin
if n ≤ 1 then fib=n
else fib = fib(n-1) + fib(n-2);
end; n
#comp(Seq. Srch)
#comp(Bin. Srch)
128 8
1,024 11
1,048,576 21
4,294,967,296 33
Digital Media Lab.

6. Divide-and-conquer approach
Binary Search : no overlap
Fibonacci sequence : full of overlapping
ƒ(5)
ƒ(3)
ƒ(4)
ƒ(1)
ƒ(2)
ƒ(2)
ƒ(3)
ƒ(0)
ƒ(1)
ƒ(0)
ƒ(1)
ƒ(1)
ƒ(2)
ƒ(0)
ƒ(1)
Digital Media Lab.

7. proof : Induction on n Theorem <Induction Basis> n = 2 & n = 3
<Induction Hypothesis>
Suppose for all m such that 2 ≤ m < n
T(m) > 2m/2
<Induction Step>
T(n) = T(n-1) + T(n-2) + 1
> 2(n-1)/2 + 2(n-2)/2 + 1
> 2(n-2)/2 + 2(n-2)/2
= 2·2(n-2)/2 = 2n/2
Digital Media Lab.

8. Algorithm 2 (Iteration)
function fib2 (n : integer) : integer;
var f : array[0..n] of integer;
i : index;
begin (n+1) terms for fib2(n)
f[0] = 0;
if n > 0 then
f[1] = 1;
for i = 2 to n do
f[i] = f[I – 1] + f[I – 2]
fib2 = f[n]
end; n
(n+1)
Iterative Alg
Recursive Alg 40
41 ns
1048 micro s. 60
61 ns
1 s 80
81 ns
18 min
100
101 ns
13 days
120
121 ns
36 years
160
161 ns
3.8 ×107 years
200
201 ns
4 ×1013 years
Digital Media Lab.

9. Algorithm Design : techniques
(D-&-C, Dynamic Programming, Greedy, Backtracking,
Branch-&-Bound, … )
Several algorithms for the same problem.
“Which is the most Efficient algorithm?”
=> Algorithm Analysis
♧ Note : problem analysis
(solvable, unsolvable, NP-Complete, … )
Digital Media Lab.

10. Time complexity Analysis
Actual CPU time ×
# instructions × (programmers, languages, …)
→ some independent measure !
# op’s : a function of input size n
n+1, 2n/2, log n …
Input size?
(Eg) Graph 문제들: O(n+e), 소수결정문제: O(log n),
행렬곱: O(n2)…
Time complexity analysis
Determine input size
Choose the basic operation
Determine how many time the B.O. is done for each value of the input size
Digital Media Lab.

11. Example Add array Members T(n) = n (n : array size)
Matrix multiplication
T(n) = n (n : # rows) “every-case time complexity”
Sequential Search
Worst-case t.c T(n) = n = W(n)
Best-case t.c B(n) = 1 ×
Average-case t.c. A(n)
Digital Media Lab.

12. (Eg) Sorting n elements
Insertion Sort
every-case complexity ×
best-case :
worst-case :
average-case :
Quick Sort
best-case, avg-case :
“practically best but not appropriate
for real time applications.”
Merge Sort, Heap Sort
every-case t.c. :
Digital Media Lab.

13. Space complexity (memory) Complexity function
Time complexity
Space complexity (memory)
Complexity function
Note : 2 algorithms for the same prob.
1000·n , n2
Which one is faster? (or better? )
n2 > 1000·n if n > 1000
Threshold : 1000
But, theoretically 1000·n is better
Correctness analysis, Correctness proof
Digital Media Lab.

14. Order 1000·n is better than 0.01·n2 if n > 100,000
Linear-time algorithm quadratic-time algorithm
Quadratic term eventually dominates
“Throw away” low-order terms
“ algorithm” or “quadratic algorithm.”
Fig 1.3 & Table p.28 & p.29 n
0.1n2
0.1n2+n+100 10
120 20 40
160 50
250
400
100
1000
1200
100000
101100
Digital Media Lab.

15. Digital Media Lab.

16. Digital Media Lab.

17. Digital Media Lab.

18. Def) (“Big-Oh of ƒ of n”)
but
“asymptotic upper bound” : big-Oh
O(n2) “at least
as good as”
3lgn+8, 4n2
5n+7, 6n2+9
2nlgn, 5n2+2
Digital Media Lab.

19. Def) (“Omega of ƒ(n)”) but Ω(n2) “at least as bad as” 4n2 4n3+3n2
5n2+2n 2n+4n
Digital Media Lab.

20. Def) (“Theta of ƒ(n)”, “order of ƒ(n)”)
(Eg) Prove that
(proof by contradiction) Assume that
Then
This implies
Since is a constant, this can never be True. → Contradiction
3lgn n2
5n n2+9
2nlgn n2+2n
4n3+3n2
6n6+n4
2n+4n
Digital Media Lab.

21. Def) (Small Oh of ƒ(n)) (Eg) Let be given Choose Then
(By contradiction) Assume
Let Then must exist
Digital Media Lab.

22. Theorem : if , then (proof) Because Next, we prove by contradiction.
Digital Media Lab.

23. Properties of Order : (Eg) 1. 2. 3. 상수 4. 5. 6.상수 4. 5. 6.
strictly growing functions 7.
(Eg)
Digital Media Lab.

24. In practice, exact time comp. → not easy or : enough Note : instead of
Theorem
(Eg) , because
Digital Media Lab.

25. (Eg) , because (Eg) (i) , Trivial (ii) , Let be big enough
Digital Media Lab.

26. Theorem (L’Hôpital Rule)
(Eg)
Digital Media Lab.