Algorithm Analysis Bina Ramamurthy CSE116A,B

Introduction The basic mathematical techniques for analyzing algorithms are central to more advanced topics in computer science and give you a way to formalize the notion that one algorithm is significantly more efficient than another. Instead of the traditional model our text defined a model that is appropriate for today’s computations, software and hardware.

Basic Axioms Axiom 2.1: The time required to fetch an operand from memory is constant, Tfetch, and the time required to store a result in memory is a constant, Tstore. Axiom 2.2 : The times required to perform elementary arithmetic, such addition, subtraction, multiplication, division, and comparison, are all constants. These times are denoted by T+, T-, Tx, T/ and T<.

Basic Axioms (contd.) Axiom 2.3 : The time required to call a method is constant, Tcall, and the time required to return from a method is a constant, Treturn. Axiom 2.4 : The time required to store pass an argument to a method is the same as the time required to store a value in the memory Tstore.

Examples y = x; y = 1; y = y + 1; y++; y = f(x);

A Simple Example public class Example { public static int sum (int n) int result = 0; for (int i= 1; i<= n; i++) result += I; return result; }

Array Access Axiom 2.5 : The time required for the address calculation implied by an array operation, for example, a[i] is a constant, T[.] . This time does not include the time to compute the subscript expression, nor does it include the time to access (fetch) the array element. Example : y = a [k]; 3Tfetch + T[.] + Tstore

Example : Horner’s Rule Horner’s rule gives a method to evaluate polynomials.  ai x^i public static int horner (int[], int n, int x) { int result = a [n]; for (int j = n-1; j > = 0; j--) result = result * c + a[j]; return result; }

Example: findMaximun public static int findMaximum(int [] a) { int result = a[0]; for (int j = 1; j < a.length; j++) if (a[j] > result) result = a[j]; return result; }

Average Running Times In the last two examples we computed running times as a function of number of input values and the actual input values. What will be average running time? If we run the program with various sequences of n numbers what will the average running times be?

Order-of-Magnitude Analysis If Alg A requires time proportional to f(N), Alg A is said to be order f(N), which is denoted by O(f(N)); f(N) is called the algorithm’s growth-rate function. The notation uses the upper-case O to denote order, it is called the Big O notation. If a problem size of N requires time that is directly proportional to N, the problem is O(N), if it is , then it is O( ), and so on.

Key concepts Formal definition of the order of an algorithm: Algorithm A is order f(N)-- denoted O(f(N)) -- if constants c and N0 exist such that A requires no more than c* f(N) time units to solve a problem of size N >= N0.

Interpretation of growth-rate functions 1 -- A growth rate function 1 implies a problem whose time requirement is constant and, therefore independent of problem size. log2N -- Time requirement for a logarithmic algorithm increases slowly as the problem size increases. For example, if you square the problem size you only double the time requirement. N -- Linear algorithm: Time requirement increases directly with the size of the problem. N-squared: Quadratic. Algorithms that use two nested loops are examples.

Interpretation of growth-rate functions N-cubed : Time requirement for a cubic algorithm increases more rapidly. Three nested loops is an example. 2-power N : exponential algorithm. Too rapidly to be of any practical use. N * log N : Algorithms that divide the problems into subproblems and solve them. N-squared: Quadratic. Algorithms that use two nested loops are examples.

Properties of growth rate functions You can ignore low-order terms in an algorithm’s growth: Example: You can ignore multiplicative constant in the higher order term of a growth-rate function: You can combine growth rate functions. Example: O(f(N))) + O(g(N)) = O(f(N) + g(N))

Worst-case and average-case analyses A particular algorithm may require different times to solve different problems of the same size. For example: searching for an element in a sorted list. Worst-case analysis gives the pessimistic time estimates. Average-case analysis attempts to determine the average amount of time that A requires to solve the problems of size N.

How to use Order-Of-Magnitude function? For example, array-based listRetrieve is O(1) : meaning whether it is nth element or 1st element it will take the same time to access it. A linked-list based listRetrieve is O(N) : meaning that the retrieval time depends on the position of the element in the list. When using an ADT’s implementation, consider how frequently particular ADT operations occur in a given application.

How to …? If the problem size is small, you can ignore an algorithm’s efficiency. Compare algorithms for both style and efficiency. Order-of-magnitude analysis focuses on large problems. Sometimes you may have to weigh the trade-offs between an algorithm’s time requirements and its memory requirements.

Efficiency of search algorithms Linear search: (sequential search) : Best case : First element is the required element: O(1) Worst case: Last element or element not present : O(N) Average case: N/2 : After dropping the multiplicative constant (1/2) : O(N)

Binary search algorithm Search requires the following steps: 1. Inspect the middle item of an array of size N. 2. Inspect the middle of an array of size N/2 3. Inspect the middle item of an array of size N/power(2,2) and so on until N/power(2,k) = 1. This implies k = log2N k is the number of partitions.

Binary search algorithm Best case : O(1) Worst case : O(log2N) Average Case : O(log2N)/2 = O(log2N)

Efficiency of sort algorithms We will consider internal sorts (not external sorts). Selection sort, Bubble sort(exchange sort), Insertion sort, Merge sort, Quick sort, Radix sort For each sort, study the 1. Algorithm 2. Analysis and Order of magnitude expression 3. Application