Reference: Tremblay and Cheston: Section 5.1 T IMING A NALYSIS

Quality of a project/method is determined by many factors: Simplicity Readability Verifiability Correctness: yields the correct results according to its specification Validity Solves the original problem Robustness Ability to handle errors Modifiability Reusability Portability Ease in moving the project/method to another machine, system, language, etc Integrity Able to withstand unauthorized access Efficiency Time Space In this segment, the focus is on measuring time efficiency. It is an important measure of quality, but certainly not the most important.

Time efficiency estimate the order of magnitude for the time requirements usually expressed a rate of growth The rate of growth is expressed as a function that is dependent upon the size of the problem, the amount of data, and/or the size of structures involved. Example public static int count(String[ ] a, String s) { int count = 0; int i = 0; while (i < a.length) { if (s.equals(a[i])) count = count + 1; i = i + 1; } return count; }

I. Statement count approach If all statements take about the same amount of time then count the number of statements executed and use it as a measure of the amount of time required to execute the algorithm. Usually a reasonable approach, provided that there are no method calls. For the count method, what determines the size of the problem? how many statements are executed?

Size of the problem for the count method: size of the array Let n = the size of the array Let T count (n) = number of statements executed in running method count on an array of size n T count (n) = 3n + q + 4 where q equals the number of times “count = count + 1” is done Best case: q = 0 T count B (n) = 3n + 4 Worst case: q = n T count W (n) = 4n + 4 Average/expected case: ??

Notation Definition: O(n) = { function t |  c, n 0 >0, such that t(n) ≤ c*n,  n ≥ n 0 } Intuitively: O(n) is the set of functions which grow linearly or less than linearly O(n) = { n, 5n, 100n , log(n), 2n + 500*log(n), (log(n)) 2, 5, , … } Note that the constant on the fastest growing term doesn’t matter and terms with slower growth don’t matter. Note that T count (n)  O(n) Alternate notation: T count (n) = O(n) Alternate definition: t(n)  O(n) if lim t(n) <  n  n Note that L’Hopitals rule is often needed if f(n)  and g(n)  as n , then lim n  f(n)/g(n) = lim n  f (n)/g (n)

II. Statement count approach Determine an operation that is done is often as any other operation (within a constant factor) and is central to the algorithm. This operation is called an active operation. Count the number of times that this operation is done, and use it as a measure of the order of magnitude of the time requirements of the algorithm. What makes a good active operation for the count method?

The active operation for the count method is s.equals(a[i]) Let K(n) = number of active operations done by the count method for an array of size n It is easy to see that K(n) = n Note that K(n)  O(n) The active operation approach is usually much easier and yields the same result. The key is the selection of the active operation. If it isn’t clear what is the active operation, include the count for each possible active operation and add them together. To sure to include the active operations in methods called.

Summation notation: Reference Appendix C.1 of text  i=1 n x i = x 1 + x 2 + x 3 + … + x n by definition Summation manipulation:  i=1 n x i =  i=1 k-1 x i + x k + x k+1 +  j=k+2 n x j  i=1 n (2x i + 3) =  i=1 n 2x i +  i=1 n 3 = 2*  i=1 n x i +  i=1 n 3 = 2*  i=1 n x i + 3*n Useful summation formulae:  i=1 n i = n(n+1)/2  i=1 n i 2 = n(n+1)(2n+1)/6  i=0 n a i = (a n+1 – 1)/(a – 1) for a ≠ 1  i=1 n i*a i = na n+1 /(a-1) + a(a n – 1)/(a – 1) 2 for a ≠ 1 (each of these can be proved by mathematical induction)

Insertion sort algorithm General algorithm: For each item to be sorted insert it into its proper place relative to previous items i.e., move larger items one position further from the front public static > void insertionSort(T[ ] a) { int i = 1; while (i < a.length) { T temp = a[i]; int j = i - 1; while (j >= 0 && temp.compareTo(a[j]) < 0) { a[j+1] = a[j]; j = j - 1; } a[j+1] = temp; i = i + 1; }

Timing analysis using statement count approach: Let T insort W (n) = worst case time for the insertionSort algorithm for an array of size n T insort W (n) =  i=1 n -1 (  j=i -1 0 (Decreasing) ( ) ) = 2 +  i=1 n -1 ( 4 +  j=0 i ) = 2 +  i=1 n -1 (6 + 3*i) = 2 +  i=1 n  i=1 n -1 3*i = 2 + 6*(n-1) + 3*  i=1 n -1 i = 2 + 6*(n-1) + 3*(n-1)(n)/2 = 2 + 6n /2*n 2 - 3/2n = 1.5 n n – 4 Note that T insort W (n)  O(n)

Definition For f(n) : I +  R + where I + is the set of positive integers and R + is the set of positive real values O(f(n)) = { function t |  c, n 0 >0, such that t(n) ≤ c*f(n),  n ≥ n 0 } Intuitively: O(f(n)) is the set of functions which grow no faster than f(n). If t(n)  O(f(n)), then t(n) is in the order of f(n), or more simply, t(n) is order f(n). Alternate definition: t(n)  O(f(n)) if lim n  t(n)/f(n) <  T insort W (n) = 1.5 n n – 4  O(n 2 ) 50 n  O(n 2 ) n*log(n)  O(n 2 ) (log(n)) j  O(n) for j <  n 5  O(2 n ) 2 n  O(n!) n!  O(n n )

Definition For f(n) : I +  R + where I + is the set of positive integers and R + is the set of positive real values  (f(n)) = { function t |  b, c, n 0 >0, such that b*f(n) ≤ t(n) ≤ c*f(n),  n ≥ n 0 } If t(n)   (f(n)), then t(n) is in the exact order of f(n). Alternate definition: t(n)   (f(n)) if lim n  t(n)/f(n) = c, 0 < c <  T count W (n) = 4n + 4   (n) T insort W (n)   (n 2 ) T count W (n)   (n 2 ) Often = and ≠ are used instead of  and .

Action operation analysis of insertion sort algorithm Active operation j >= 0 && temp.compareTo(a[j]) < 0 T insort W (n) =  i=1 n -1 ( 1 +  j=i -1 0 (Decreasing) (1)) =  i=1 n -1 ( 1 + i ) = (n – 1) + (n – 1)(n)/2 = n 2 /2 + n/2 – 1   (n 2 ) T insort B (n) =  i=1 n -1 ( 1 ) = n – 1   (n) T insort E (n) =  i=1 n -1 ( 1 + (1/2)  j=i -1 0 (Decreasing) (1)) assuming usually insert ½ way back ~ n 2 /4 + 3n/4   (n 2 )

Does rate of growth make a difference, or are computers so fast that any algorithm can be done quickly? Consider how large a problem can be solved in one minute using algorithms of different time complexity: Linear algorithm: 1,000,000,000 Quadric algorithm: 50,000 Factorial algorithm (n!): 11 Some algorithms take huge amounts of time on even very small problems. Even the difference between linear and quadratic can make a big difference.

Combining growth functions For sets A and B and element c, define the following: A + B = { a + b | a  A and b  B} A*B = { a * b | a  A and b  B} c * B = { c * b | b  B} Using these operations, the following are implied: O(f(n)) + O(g(n)) = O(f(n) + g(n)) k * O(f(n)) = O(k*f(n)) = O(f(n)) for k a constant O(f(n)) + O(g(n)) = O(2*max(f(n), g(n))) = O(max(f(n), g(n)))

More elaborate example: Given int method k(int i) with time T k (m) = O(log(m)) for some m independent of i. Given void method p with time T p (n) = O(n 2 ) for some n independent of m. What is the time requirement for each of the following methods: public void q() { int c = 0; for (int i = 1; i < n + 1; i++) c = c + k(i); } public void r() { q(); for (int i = 1; i < m; i = 2*i) p(); s(); } public void s() { int x = 0; for (int i = 0; i < m + 1; i++) x = x + k(i)*k(2*m - i); System.out.println(x); }

Analysis of method q: active operation: c = c + k(i) number of times that it is done: n cost of doing the active operation: O(log(m)) total cost: n*O(log(m)) = O(n*log(m)) Analysis of method s: active operation: x = x + k(i)*k(2*m - i) number of times that it is done: m + 1 cost of doing the active operation: 2*O(log(m)) total cost: (m + 1)*2*O(log(m)) = O(m*log(m))

T q (n, m) = n*O(log(m)) = O(n*log(m)) T s (m) = m*2*O(log(m)) = O(m*log(m)) Analysis of method r: Three possible active operations, so analyze all three: active operation: q() number of times that it is done: 1 cost of doing the active operation: O(n*log(m)) cost for q: O(n*log(m)) active operation: s() number of times that it is done: 1 cost of doing the active operation: O(m*log(m)) cost for s: O(m*log(m)) active operation: p(); number of times that it is done: ?? cost of doing the active operation: O(n 2 ) cost for p: ?? * O(n 2 ) Total cost for r: cost for active op q + cost for active op s + cost for active op p T r (n, m) = O(n*log(m)) + O(n*log(m)) + ?? * O(n 2 )

How many times is the loop in method r done? Let x = number of times that the loop in method r is done Consider the value of i 1, 2, 4, 8, 16, …, 2 x-1, 2 x Consider when the loop is exited; when i  m; i.e., 2 x  m 2 x-1 < m  2 x x-1 < log 2 (m)  x (taking the log to the base 2 of each term) x is log 2 (m) or the next integer value larger than it Therefore x =  log 2 (m)  Notation:  p  = smallest integer greater than or equal to p, called the ceiling function  p  = largest integer less than or equal to p, called the floor function

T r (n, m) = T q (n, m) +  log 2 (m)  *T p (n) + T s (m) = O(n*log(m)) +  log 2 (m)  *O(n 2 ) + O(m*log(m)) = O(n*log(m) + n 2 *log(m) + m*log(m)) = O(n 2 *log(m) + m*log(m)) = O(max(n 2, m)*log(m))

Example What is the time complexity of the method? How many times is the loop done? Let r = number of times that the loop is done static public int f(int n) { int result = 0; double p =n; while (p > 1) { p = p/2; result = result + 1; } return result; }

Consider the values of p n, n/2, n/4, n/8, …, n/2 r-1, n/2 r <= 1 When the method finishes n/2 r <= 1 < n/2 r-1 i.e, n <= 2 r and 2 r-1 < n and taking log to the base 2 of each term r – 1 < log 2 (n) <= r Therefore, since r is an integer r =  log 2 (n)  Thus, f is the function  log 2 (n)  for n>= 1, and T f (n) =  (  log 2 (n)  ).