Algorithm Complexity Level II, Term ii CSE – 243 Md. Monjur-ul-hasan Lecturer Dept of cse, cuet Email: mail@monjur-ul-hasan.info

A motivation for complexity Talking about running time and memory space of algorithms is difficult. Different computers will run them at different speeds and memory Different problem sizes run for a different amount of time and memory(even on same computer). Different inputs of the same size run for a different amount of time and memory. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Definition: Complexity Efficiency or complexity of an algorithm is stated as a function relating the input length to the number of steps (time complexity) or storage locations (space complexity). log n n n log n n2 n3 2n 1 2 4 8 16 64 3 24 512 256 4096 65536 5 32 160 1024 32768 4294967296 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

More Curve exp (n) n Fig 1.3 Fundamental Computer Algorithm f(n) By, log n exp (n) f(n) More Curve Fig 1.3 Fundamental Computer Algorithm By, E. Horowitz S. Sahni S. Rajasekaran Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Using Upper and Lower Bounds to talk about running time. Problem Size Running time (and memory) on some computer 5 10 15 20 Different inputs Upper Bound Lower Bound Md. Monjur-ul-hasan. Lecturer, Dept of CSE CUET

Bounds on running time We want to bound the running time: “for arrays of size n it takes my computer 45·n2 milliseconds at most to sort the array with bubblesort” But what about other computers? (faster/slower) Instead we say: “On any computer, there is a constant c such that it takes at most c·n2 time units to sort an array of size n with bubblesort” How is this sentence useful? Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Big O notation. We say that running time of an alg. is O(f(n)) If there exists c such that for large enough n: RunningTime(n) < c·f(n) Translation: From some point c·f(n) is an upper bound on the running time. Why only for “large enough n”? Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Big-O notation Do not assume that because we analyze running time asymptotically we no longer care about the constants. If you have two algorithms with different asymptotic limits – easy to choose. (may be misleading if constants are really large). Usually we are happy when something runs in polynomial time and not exponential time. O(n2) is much much better than O(2n). Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Use of big-O notation Example: The running time of bubble sort on arrays of size n is O(n2). Translation: for a sufficiently large array, the running time is less than c·n2 for some c (No matter how bad the input is). Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Big Omega For lower bounds we use Ω(f(n)). I.e., if we say the running time of an algorithm is Ω(f(n)), we mean that RunningTime(n)>c·f(n) for some c. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

An Example: Checking a the number n is prime. We can try and divide the number n by all integers in the range 2,…,n1/2. Best case: sometimes we find it isn’t prime really quickly (e.g. it divides by 2). Worst case: sometimes the number is prime and we try n1/2 different divisions. Running time is O(n1/2), and is also Ω(1). Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Bounds might not be tight We saw that finding out if a number n is prime takes us O(n1/2) time. It also takes us O(n5) time. The second bound guarantees less. It is not tight. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Big Theta Notation The function f(n) = Θ(g(n)) iff there exists positive constants c1 and c2 such that c1Xg(n) < f(n) < c2Xg(n). Average time Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Measurement: Tabular Method Algorithm Algorithm Sum(a,n) { sum := 0.0 for i:= 1 to n do sum := sum +a[i]; return sum } Significant Frequency Total 1 - n+1 n 2n+3 Complexity: O(n) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

Bounds might not be tight Algorithm Algorithm Add(a,b,c,m,n) { for i:= 1 to m do for j:=1 to n do c[I,j] := a[i,j] + b[i,j]; } Significant Frequency Total 1 - m+1 n+1 n mn+m mn 2mn+2m+1 Complexity: O(mn) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET