1 Efficiency of Algorithms

2  Two main issues related to the efficiency of algorithms:  Speed of algorithm  Efficient memory allocation  We will focus on the speed of algorithms. The speed of an algorithm (running time) is determined by the number of elementary operations: addition, subtraction, multiplication, division, comparison. The number of elementary operations depends on  problem size  nature of input data

3 Analysis of Running Time  Running time of an algorithm is a function of input size.  Two kind of analyses of running time:  Worst-case running time analysis  Average-case running time analysis Most results in the analysis of algorithms concern the worst-case running time.

4 Efficiency of Algorithms: Example Traveling Salesman Problem (TSP)  Recall the Traveling Salesman Problem (TSP) : There are n cities. The salesman  starts his tour from City 1,  visits each of the cities exactly once,  and returns to City 1. For each pair of cities i,j there is a cost c ij associated with traveling from City i to City j. Goal: Find a minimum-cost tour.

5 “Exhaustive enumeration” algorithm  One way of solving the problem is by an algorithm which is based on exhaustive enumeration (brute force): 1) Compute the costs of all possible tours; 2) Choose the tour with minimum cost.  What is the running time of this algorithm? 1)  The number of possible tours is (n-1)!  The cost of each tour is the sum of n individual costs  requires n-1 additions.  Thus, computing the costs of all possible tours requires (n-1)∙(n-1)! elementary operations. 2) Choosing the tour with minimum cost requires (n-1)!-1 comparisons. Summarizing, (n-1)∙(n-1)! + (n-1)! -1 = n! -1 elementary operations are needed for implementing the algorithm.

6 Efficiency of the “Exhaustive enumeration” algorithm Is the “exhaustive enumeration” algorithm efficient? Assume that each elementary operation can be done in 1 nanosecond = seconds. Then the running time: n=10n=20n= sec77 years 8.4  centuries

7 “Nearest neighbor” algorithm  Consider another method for solving the TSP, the “Nearest neighbor” algorithm: In every iteration (except the last one) go to the closest city not visited yet.  What is the running time of this algorithm?  In each iteration we choose one of the n cities; so there are n iterations.  In each iteration, we need at most n comparisons to choose the next city. Thus, the total number of elementary operations is at most n 2.

8 Efficiency of the “Nearest Neighbor” algorithm Compare the running times of “Nearest neighbor” and “Exhaustive enumeration” algorithms: Note: The “Nearest Neighbor” algorithm might not return an optimal solution. n=10n=30n=100n=1000 n2n sec sec sec sec. n!0.004 sec 8.4  cent.

9 Bad case example for Bad case example for the “Nearest Neighbor” algorithm Output of the “Nearest neighbor” algorithm Optimal solution

10 Overwhelming terms in running times Suppose in the “Exhaustive enumeration” algorithm, we need to preprocess the costs before applying the main routine (e.g., converting kilometers to miles). Preprocessing requires at most n 2 multiplications; thus, the running time of the new algorithm is n!+n 2. For large n, the term n! overwhelms the term n 2. Thus, the running time n!+n 2 qualitatively is not much different from n!.

11 Order of an Algorithm  Definition: Let A be an algorithm. Let w(n) be the maximum number of elementary operations required to execute A for all possible input sets of size n. If w(n) is O(f(n)), we say that A has a (worst case) order of f(n).  Ex.: Suppose the maximum number of operations needed to execute algorithm A is 5n 2 +3n+7. Then A has an order of n 2.  Definition: An algorithm is called polynomial-time if it has an order of f(n)=n k for some constant k.  Polynomial-time algorithms are normally considered efficient.

Time comparisons of the most common algorithm orders f(n)n=10n=1000n=10 5 n=10 7 log 2 n 3.3  sec sec. 1.7  sec.2.3  sec. n sec sec sec.0.01 sec. n∙ log 2 n 3.3  sec sec sec.0.23 sec. n2n sec sec.10 sec.27.8 min. n3n sec.1 sec.11.6 min.317 cent. 2n2n sec. 3.4  years 3.2  years 3.1  years