1 Section 2.3 Complexity of Algorithms

2 Computational Complexity Measure of algorithm efficiency in terms of: –Time: how long it takes computer to solve problem with given number of inputs –Space: amount of memory required to implement an algorithm for given number of inputs Space complexity involves data structures in use - we will limit discussion to time complexity

3 Time Complexity Described in terms of number of operations required because different computers, or same computer under different conditions, take different amounts of time to perform same basic operation

4 Complexity of Summation Algorithm Basic operations: –addition –comparison (to see if end of list has been reached) Each operation must be performed N times for N elements –So total of 2N operations performed when algorithm applied –time complexity is O(N)

5 Complexity of Linear Search Basic operation is comparison; 2 performed at each step: –determine if item found –determine if list exhausted Worst-case: item not found –whole list must be searched to determine this - total of 2N + 2 operations –so linear search is O(N)

6 Complexity of Binary Search Begin with assumption that N (number of input elements) is a number divisible by 2 The N = 2 k, and k = log 2 N At each stage of algorithm, left & right indexes are compared - if left < right, middle index is calculated and target is compared to element at middle

7 Complexity of Binary Search First time through loop, 2 comparisons are used to limit search area to 2 k-1 items Next time through, 2 more comparisons are performed, and search area is reduced to 2 k- 2 items Eventually, list search area is reduced to 2 0 elements and one comparison is required to determine if list item is target

8 Complexity of Binary Search So binary search algorithm requires, at most, 2k + 2 or 2(log 2 N) + 2 comparisons for a list of 2 k elements Time complexity is O(log N)

9 Worst-Case Analysis Number of operations required to guarantee an algorithm produces a solution Since log N is a smaller number than N, and grows much more slowly than N, binary search is more efficient than linear search in the worst case

10 Commonly Used Terminology for Complexity of Algorithms O(1)Constant complexity O(log N)Logarithmic complexity O(N)Linear complexity O(N log N)N log N complexity O(N b )Polynomial complexity O(b N )Exponential complexity (where b > 1) O(N!)Factorial complexity

11 Tractability A problem that is solvable using an algorithm of polynomial complexity (or less) is said to be tractable This means there is an expectation that a solution can be found in a reasonable amount of time Generally true if degree and coefficient of polynomial are relatively small

12 Tractability An intractable problem is one that cannot be solved with worst-case polynomial time In practice, an intractable problem is not necessarily unsolvable; remember, we’re only looking at the worst case An unsolvable problem is one for which no algorithm exists

13 Limitations of Big-O Estimate of time complexity of algorithm expresses how the amount of time required to solve a problem changes with larger and larger input sets Can’t be directly translated into amount of time used by algorithm

14 Comparisons of Actual Computer Time for Algorithms The next slide presents a table of actual time required to solve problems of specific sizes given these assumptions: –All operations reduced to bit level, so algorithm complexity translates directly to number of bit operations –Each operation requires exactly seconds to perform

15 Computer Time Used by Algorithms

16 Notes on Computer Times Most algorithms, even O(N 2 ), execute in a reasonable amount of time for small inputs But note difference as N grows for quadratic algorithm: –for N=10,000, takes.1 second –for N=100,000, takes 10 seconds –for N=1,000,000, takes 17 minutes The difference grows even faster for O(2 N ) and O(N!)

17 Section 2.3 Complexity of Algorithms -ends-