Asymptotic Growth Rates  Themes  Analyzing the cost of programs  Ignoring constants and Big-Oh  Recurrence Relations & Sums  Divide and Conquer  Examples  Sort  Computing powers  Euclidean algorithm (computing gcds)  Integer Multiplication

Asymptotic Growth Rates – “Big-O” (upper bound)  f(n) = O(g(n)) [f grows at the same rate or slower than g] iff: There exists positive constants c and n 0 such that f(n)  c g(n) for all n  n 0  f is bound above by g  Note: Big-O does not imply a tight bound  Ignore constants and low order terms

Big-O, Examples  E.G. 1: 5n 2 = O(n 3 ) c = 1, n 0 = 5: 5n 2  n  n 2 = n 3  E.G. 2: 100n 2 = O(n 2 ) c = 100, n 0 = 1  E.G. 3: n 3 = O(2 n ) c = 1, n 0 = 12 n 3  (2 n/3 ) 3, n  2 n/3 for n  12 [use induction]

Little-o Loose upper bound  f(n) = o(g(n)) [f grows strictly slower than g]  f(n) = O(g(n)) and g(n)  O(f(n))  lim n  f(n)/g(n) = 0  “f is bound above by g, but not tightly”

Little-o, restatement  lim n  f(n)/g(n) = 0  f(n) = o(g(n))    >0,  n 0 s.t.  n  n 0, f(n)/g(n) < 

Equivalence - Theta  f(n) =  (g(n)) [grows at the same rate]  f(n) = O(g(n)) and g(n) = O(f(n))  g(n) =  (f(n))  lim n  f(n)/g(n) = c, c ≠ 0  f(n) =  (g(n))  “f is bound above by g, and below by g”

Common Results  [j < k] lim n  n j /n k = lim n  1/n (k-j) = 0  n j = o(n k ), if j<k  [c < d] lim n  c n /d n = lim n  (c/d) n = 0  c n = o(d n ), if c<d  lim n  ln(n)/n =  /   lim n  ln(n)/n = lim n  (1/n)/1 = 0 [L’Hopital’s Rule]  ln(n) = o(n) [  > 0] ln(n) = o(n  ) [similar calculation]

Common Results  [c > 1, k an integer] lim n  n k /c n =  /   lim n  kn k-1 / c n ln(c)  lim n  k(k-1)n k-2 / c n ln(c) 2  …  lim n  k(k-1)…(k-1)/c n ln(c) k = 0  n k = o(c n )

Asymptotic Growth Rates   (log(n)) – logarithmic [log(2n)/log(n) = 1 + log(2)/log(n)]   (n) – linear [double input  double output]   (n 2 ) – quadratic [double input  quadruple output]   (n 3 ) – cubit [double input  output increases by factor of 8]   (n k ) – polynomial of degree k   (c n ) – exponential [double input  square output]

Asymptotic Manipulation   (cf(n)) =  (f(n))   (f(n) + g(n)) =  (f(n)) if g(n) = O(f(n))

Computing Time Functions  Computing time function is the time to execute a program as a function of its inputs  Typically the inputs are parameterized by their size [e.g. number of elements in an array, length of list, size of string,…]  Worst case = max runtime over all possible inputs of a given size  Best case = min runtime over all possible inputs of a given size  Average = avg. runtime over specified distribution of inputs

Analysis of Running Time  We can only know the cost up to constants through analysis of code [number of instructions depends on compiler, flags, architecture, etc.]  Assume basic statements are O(1)  Sum over loops  Cost of function call depends on arguments  Recursive functions lead to recurrence relations

Loops and Sums  for (i=0;i<n;i++)  for (j=i;j<n;j++)  S; // assume cost of S is O(1)

Merge Sort and Insertion Sort  Insertion Sort  T I (n) = T I (n-1) + O(n) =  (n 2 ) [worst case]  T I (n) = T I (n-1) + O(1) =  (n) [best case]  Merge Sort  T M (n) = 2T M (n/2) + O(n) =  (nlogn) [worst case]  T M (n) = 2T M (n/2) + O(n) =  (nlogn) [best case]

Karatsuba’s Algorithm  Using the classical pen and paper algorithm two n digit integers can be multiplied in O(n 2 ) operations. Karatsuba came up with a faster algorithm.  Let A and B be two integers with  A = A 1 10 k + A 0, A 0 < 10 k  B = B 1 10 k + B 0, B 0 < 10 k  C = A*B = (A 1 10 k + A 0 )(B 1 10 k + B 0 ) = A 1 B k + (A 1 B 0 + A 0 B 1 )10 k + A 0 B 0 Instead this can be computed with 3 multiplications  T 0 = A 0 B 0  T 1 = (A 1 + A 0 )(B 1 + B 0 )  T 2 = A 1 B 1  C = T k + (T 1 - T 0 - T 2 )10 k + T 0

Complexity of Karatsuba’s Algorithm  Let T(n) be the time to compute the product of two n-digit numbers using Karatsuba’s algorithm. Assume n = 2 k. T(n) =  (n lg(3) ), lg(3)  1.58  T(n)  3T(n/2) + cn  3(3T(n/4) + c(n/2)) + cn = 3 2 T(n/2 2 ) + cn(3/2 + 1)  3 2 (3T(n/2 3 ) + c(n/4)) + cn(3/2 + 1) = 3 3 T(n/2 3 ) + cn(3 2 / /2 + 1) …  3 i T(n/2 i ) + cn(3 i-1 /2 i-1 + … + 3/2 + 1)...  cn[((3/2) k - 1)/(3/2 -1)] --- Assuming T(1)  c  2c(3 k - 2 k )  2c3 lg(n) = 2cn lg(3)

Divide & Conquer Recurrence Assume T(n) = aT(n/b) +  (n)  T(n) =  (n) [a < b]  T(n) =  (nlog(n)) [a = b]  T(n) =  (n log b (a) ) [a > b]

(blank for notes)