Presentation is loading. Please wait.

Presentation is loading. Please wait.

Foundations of Algorithm

Similar presentations


Presentation on theme: "Foundations of Algorithm"— Presentation transcript:

1 Foundations of Algorithm
算法基础 Foundations of Algorithm

2 Outline What is algorithm? Time complexity analysis Divide and Conquer

3 What is Algorithm?

4 What is Algorithm?

5 What is Algorithm?

6 What is Algorithm? Pseudocode

7 Problem : A question to which we seek an answer. Answer Definition:
Algorithm_Lecture Problem : A question to which we seek an answer. Answer Definition: A step-by-step procedure → Algorithm (Eg) problem : Determine whether instance : answer → yes Algorithm1 : Sequential Search Procedure seqsearch ( n : integer; S : array[1..n] of keytype x : keytype; var location : index); begin location := 1; while location ≤ n and S[location] ≠ x do location := location + 1; if location > n then location := 0; end; 5 7 8 10 11 13 Digital Media Lab.

8 Algorithm2 : Binary Search
Procedure binsearch ( n : integer; S : array[1..n] of keytype; x : keytype; var location : index); var low, high, mid : index; begin low = 1; high = n; location = 0; while low ≤ high and location = 0 do mid = ( low + high ) / 2; if x = S[mid] then location = mid; else if x < S[mid] then high = mid – 1; else low = mid + 1; end;

9 Algorithm2 : Binary Search
5 7 8 10 11 13 middle 8 11>8 10 11 middle 11 Find!

10 Algorithm2 : Binary Search
Procedure binsearch ( n : integer; S : array[1..n] of keytype; x : keytype; var location : index); var low, high, mid : index; begin low = 1; high = n; location = 0; while low ≤ high and location = 0 do mid = ( low + high ) / 2; if x = S[mid] then location = mid; else if x < S[mid] then high = mid – 1; else low = mid + 1; end; 2 different techniques for same problem → Efficiency? time : space :

11 Fibonacci Sequence Algorithm 1 (Recursion)
function fib (n : integer) : integer; begin if n ≤ 1 then fib=n else fib = fib(n-1) + fib(n-2); end; n #comp(Seq. Srch) #comp(Bin. Srch) 128 8 1,024 11 1,048,576 21 4,294,967,296 33

12 proof : Induction on n Theorem <Induction Basis> n = 2 & n = 3
<Induction Hypothesis> Suppose for all m such that 2 ≤ m < n T(m) > 2m/2 <Induction Step> T(n) = T(n-1) + T(n-2) + 1 > 2(n-1)/2 + 2(n-2)/2 + 1 > 2(n-2)/2 + 2(n-2)/2 = 2·2(n-2)/2 = 2n/2

13 Algorithm 2 (Iteration)
function fib2 (n : integer) : integer; var f : array[0..n] of integer; i : index; begin (n+1) terms for fib2(n) f[0] = 0; if n > 0 then f[1] = 1; for i = 2 to n do f[i] = f[I – 1] + f[I – 2] fib2 = f[n] end; n (n+1) Iterative Alg Recursive Alg 40 41 ns 1048 micro s. 60 61 ns 1 s 80 81 ns 18 min 100 101 ns 13 days 120 121 ns 36 years 160 161 ns 3.8 ×107 years 200 201 ns 4 ×1013 years

14 Algorithm Design : techniques
(D-&-C, Dynamic Programming, Greedy, Backtracking, Branch-&-Bound, … ) Several algorithms for the same problem. “Which is the most Efficient algorithm?” => Algorithm Analysis ♧ Note : problem analysis (solvable, unsolvable, NP-Complete, … ) =𝑎×𝑏? 3607×3803= ?

15 Time complexity Analysis
Actual CPU time × # instructions × (programmers, languages, …) → some independent measure ! Condition: a function of input size n n+1, 2n/2, log n … Time complexity analysis Determine input size Choose the basic operation Determine how many time the B.O. is done for each value of the input size

16 Time complexity Space complexity (memory) Complexity function Note : 2 algorithms for the same prob. 1000·n , n2 Which one is faster? (or better? ) n2 > 1000·n if n > 1000 Threshold : 1000 But, theoretically 1000·n is better Correctness analysis, Correctness proof

17 Order 1000·n is better than 0.01·n2 if n > 100,000
Linear-time algorithm quadratic-time algorithm Quadratic term eventually dominates “Throw away” low-order terms “ algorithm” or “quadratic algorithm.” n 0.1n2 0.1n2+n+100 10 120 20 40 160 50 250 400 100 1000 1200 100000 101100

18 Digital Media Lab.

19 Digital Media Lab.

20 Digital Media Lab.

21 Def) (“Big-Oh of ƒ of n”)
but “asymptotic upper bound” : big-Oh O(n2) “at least as good as” 3lgn+8, 4n2 5n+7, 6n2+9 2nlgn, 5n2+2

22 Def) (“Omega of ƒ(n)”) but Ω(n2) “at least as bad as” 4n2 4n3+3n2
5n2+2n 2n+4n .

23 Def) (“Theta of ƒ(n)”, “order of ƒ(n)”)
(Eg) Prove that (proof by contradiction) Assume that Then This implies Since is a constant, this can never be True. → Contradiction 3lgn n2 5n n2+9 2nlgn n2+2n 4n3+3n2 6n6+n4 2n+4n

24 Def) (Small Oh of ƒ(n)) (Eg) Let be given Choose Then
(Proof by contradiction) Assume Let Then must exist

25 o(n2) “asymptotically negligible”
Theorem : if , then (proof) Because Next, we prove by contradiction. if , then Because 4n3+3n2 6n6+n4 2n+4n 3lgn n2 5n n2+9 2nlgn n2+2n o(n2) “asymptotically negligible”

26 In practice, exact time comp. → not easy or : enough Note : instead of
Theorem (Eg) , because

27 (Eg) , because (Eg) (i) , Trivial (ii) , Let be big enough

28 Theorem (L’Hôpital Rule)
(Eg)

29 Divide-and-Conquer (i) Divide into ≥ 2 smaller instances
(ii) Solve each instance (Recursively) (iii) Combine the subsolutions. (Eg) merge sort, Quick sort, … Top-down : more natural iteration : faster

30 Binary Search : locate x in a sorted array.
Strategy (or approach) If x equals S[mid], return with mid. 1. Divide into 2 subarrays. If x < s[mid], left subarray. Right subarray otherwise. 2. Conquer(Solve) the chosen subarray. Unless sufficiently small, use recursion. 3. Obtain the solution from the subsolution. (Eg) x= S : Choose L.S.A Because x<20 mid Choose R.S.A Because x>13 mid 14 18 mid 18 return (mid) mid

31 function location (low, high : index) : index;
var mid : index; begin if low > high then location = 0; else mid = (low + high) / 2; if x = S[mid] then location = mid; else if x < S[mid] then location = location(low, mid – 1); else location = location(mid + 1, high); end; n, S, x : global variables. Why? (ugly) In implementation of recursive routine, call-by-value in each rec. call. unchanging variables : parameters ×

32 Recursion : more natural, concise, clear, …
Call-by-address? No. Confusing. Another technique – good! procedure binsrch2 (n : integer; S : array [1..n] of keytype; x : keytype) : index; var locationout : index; { function location is defined here } begin locationout = location( 1, n ); end; Recursion : more natural, concise, clear, … iteration : faster, save memory space. (because of stack manipulation) stack depth (recursion depth) :

33 No every-case time complexity
Best-case : Avg-case, worst-case : Worst-case time complexity analysis Basic operation : element comparison. ( x : S[mid] ) why? Input size : n (size of array S) (case 1) n is a power of 2

34 (case 2) n is not a power of 2

35 Merge Sort : 2-way mergesort
Note : k-way mergesort (k > 2) Strategy (or approach) 1. Divide into 2 subarrays of same size 2. Conquer (solve) each subarray. (sort) Unless sufficiently small, use recursion. 3. Combine the subsolution : merge (Example) 1. Divide the array : 2. Sort each subarray : 3. Merge the subarrays :

36

37 procedure mergesort ( n : integer;
var S : array [1..n] of keytype); const h=n/2; m=n-h; var u : array [1..h] of keytype; v : array [1..m] of keytype; begin if n > 1 then { copy S[1] ~ S[h] to u; copy S[h+1] ~ S[n] to v; mergesort ( h, u ); mergesort ( m, v ); merge ( h, m, u, v, S ); } end;

38 procedure merge ( h, m, u, v, S ) var i, j, k ; index; begin
while i ≤ h and j ≤ m do { if u[ i ] < v[ j ] then { S[ k ] = u[ i ]; i++ } else { S[ k ] = v[ j ]; j++ k++ if i > h then copy v[ j..m ] to S[ k..h+m ]; else copy u[ i..h ] to S[ k..h+m ]; end; u v i j S : k

39 worst-case time comp. Analysis of mergesort.
Basic op. : element comp. In merge Input size : n (size of S) (i) (ii)

40 In-Place Sort “Only constant extra space.” extra memory space (why?)
Reduce to n ? Yes! produce mergesort2 ( low, high : index); var mid : index; begin if low < high then { mid = ( low + high ) / 2; mergesort2( low, mid ); mergesort2( mid + 1, high ); merge2( low, mid, high ); } end;

41 procedure merge2 ( low, mid, high : index ) ;
begin i = low; j = mid + 1; k = low; while i ≤ mid and j ≤ high do { if S[ i ] < S[ j ] then { U[ k ] = S[ i ]; i++ } else { U[ k ] = S[ j ]; j++ k++ if i > mid then copy S[ j .. high ] to U[ k .. high ]; else copy S[ i .. mid ] to U[ k .. high ]; move U[ low .. high ] to S[ low .. high ]; end;

42 Quick Sort – Hoare (1962) Strategy (approach) (Eg)
Choose a pivot item (randomly) Partition into 2 subarrays. Sort each subarray recursively. (Eg) 1. Partition. 2. Sort subarrays recursively. pivot item

43 Quick Sort – Hoare (1962)

44 procedure quicksort ( low, high : index) ;
var pivotpoint : index; begin if high > low then { partition( low, high, pivotpoint ); quicksort( low, pivotpoint – 1 ); quicksort( pivotpoint + 1, high ); } end; n, S : parameters × main : quicksort (1, n );

45 Procedure partition ( low, high : index;
var pivotpoint : index ); var i, j : index; pivotitem : keytype; begin pivotitem = S[ low ]; j = low; for i = low + 1 to high do if S[ i ] < pivotitem then { j++; exchange S[ i ] & S[ j ]; } pivotpoint = j; exchange S[ low ] and S[ pivotpoint ] end;

46 In-place? Yes. If we ignore stack space.
Stable? No. Why? In-place? Yes. If we ignore stack space. (Every-case) Time complexity Analysis of Partition Basic op. : element comp.(S[i]: pivotitem) Input size : n=high-low+1 T(n)=n-1 Worst-case t.c. Analysis of Quicksort T(n)=T(0) + T(n-1) + n-1 T(n)=T(n-1)+(n-1) for n>0 T(0)=0 T(n)= T(n-1)+n-1=T(n-2)+(n-2)+(n-1)=…

47 Strassen’s Matrix Multiplication ㅡ1969
# multiplication + # additions : Better algorithm? Asymptotically yes.

48 Generally, A, B, C : n×n matrices

49 Summary What is algorithm? Time complexity analysis Divide and conquer
A step-by-step procedure Time complexity analysis Divide and conquer

50 Coin Change Problem Usable coin 968? G r e e d y 500 100 10 50 5 1 500
4 1 1 1 3 G r e e d y

51 X Coin Change Problem Problem 12? Dynamic Programming Greedy (3)
Best : 8 6 2 1 X 8 2 2 6 6 Dynamic Programming

52 { { } Divide and Conquer N[k] : #min coins for $k 0 if k == 0
N[k] = ∞ if k < 0 min if k > 0 { { } N[k - 1], N[k - 2], N[k - 6], N[k - 8] + 1


Download ppt "Foundations of Algorithm"

Similar presentations


Ads by Google