1. CS2336: Computer Science II
Today’s Agenda
Asymptotic Analysis of algorithm
A good formal mathematical understanding of algorithms running times and Big O notation
We measure algorithms and data structures against another using Asymptotic Analysis
CS2336: Computer Science II

2. CS2336: Computer Science II
Example
Suppose you write a program that manages inventory, for a nation wide chain of stores
every night you have to process all of the changes that happened in inventory in every store across the country that day!
10,000 ms to read inventory from the disk
10ms to process each transactions
for n, when n is a large number, the total time takes:
(10,000+10n) ms
for example there are 1 million transactions a day!
CS2336: Computer Science II

3. CS2336: Computer Science II
Asymptotic Analysis
Study of how algorithms behave as size of the data you processing grows very large! As it goes to infinity!
Big O notation , represent upper bound in a function. Has a very formal mathematical meaning!
CS2336: Computer Science II

4. CS2336: Computer Science II
Big O notation
Let n be size of the program’s input
size can be number of bits that our program has to read from disk for the inventory example!
it can be numbers of items, how many items I want to insert to my list!
for Asymptotic analysis we don’t care how exactly n is measured!
Let T(n) be a real world function, that describe some actual process in real.
for example an actual running time of your inventory program in ms on an input of n items.
T(n) = (10, n)
Let F(n) be another function-preferably simple function! (we want to use it as a upper bound)
CS2336: Computer Science II

5. CS2336: Computer Science II
Informal Big O
Complexity of T(n) is O( F(n) ) if and only if T(n) <= c F(n)
whenever when n is big and for some large constant c.
How big is big? choose n big enough to make T(n) function to fit under F(n)
How large the c have to be? choose c big enough to make T(n) function to fit under F(n)
CS2336: Computer Science II

6. Example T(n) = 10,000+10n lets try f(n) = n, c=20 cf(n) = 20n T(n)
30000
how they behave when n go to infinity!
T(n)<=cf(n) for any n>=1000
SO T(n) complexity is O(n)
20000
10000
1000
CS2336: Computer Science II

7. CS2336: Computer Science II
Formal Big O
O(f(n)) is an infinite set of all functions T(n) that satisfy : There exist positive constants c and N such that, for all n >= N , T(n) <= c f(n) N is 1000 in our example
CS2336: Computer Science II

8. CS2336: Computer Science II
Example
1,000,000n  O(n) Big O doesn’t care about the constant factors! n3+n2+1  O(n3) Big O is usually used to pick the fastest growing term.
CS2336: Computer Science II

9. Table of important big O sets
Function
common name
O(1)
constant
O(log n)
logarithmic
O(log 2 n )
log- squared
O(n1/2 )
root n
O(n)
linear
O(n log n)
O(n2)
quadratic
O(n3)
cubic
O(n4)
quartic
O(2n)
exponential
O(en)
exponential (but very more so!)
O(n!)
< O(nn)
Asymptotically en is much more faster than 2n
CS2336: Computer Science II

10. CS2336: Computer Science II
Logarithms
If you don’t know:
logab is equal to logcb / logca
you need to review logarithms again!
CS2336: Computer Science II

11. binary search : recursion tree
T(n) = T(n/2) + 1 = T(n/22) + 2 = T(n/23) + 3 = T(n/24) + 4 = …….. = T(n/2k) + k = T(1) + k = 1 + log2n O(log2n) if n=1 => T(1) = 1 n/2k = 1 => k = log2 n
CS2336: Computer Science II

12. A finished recursion tree
CS2336: Computer Science II

13. CS2336: Computer Science II
The Master Theorem
n is the size of the problem.
a is the number of sub-problems in the recursion.
n/b is the size of each sub-problem.
f (n) is the cost of the work done outside the recursive calls, which includes the cost of dividing the problem and the cost of merging the solutions to the sub-problems.
CS2336: Computer Science II

14. The Master Theorem
The master theorem concerns recurrence relations of the form:
f(n) = nc
1. if logb a < c, T(n) = O(nc),
2. if logb a = c, T(n) = O(nc log n),
3. if logb a > c, T(n) = O(nlogb a).
if n = 1
if n > 1

15. Example T(n) = 8T(n/2) + 1000n2 T(n) = T(n/2) + 1 T(n) = 9T(n/3) +

16. Example T(n) = 8T(n/2) + 1000n2 a = 8 , b = 2, c = 2
logba = log28= 3 > c  case 3  T(n) = O(n3)
T(n) = T(n/2) a = 1 , b = 2, c = 0
logba = log21= 0 = c  case 2  T(n) = O(n0logn) = O(logn)
T(n) = 9T(n/3) + n a = 9, b = 3, c = 1
logba = log39= 2 > c  case 3  T(n) = O(n2)
T(n) = 2T(n/2) + n a=2,b=2,c=1
logba = log22= 1 = c  case 2  T(n) = O(n1logn) = O(n logn)
T(n) = 3T(n/3) + n2 a=3,b=3,c=2
logba = log33= 1 < c  case 1  T(n) = O(n2)