CSC310 © Tom Briggs Shippensburg University Fundamentals of the Analysis of Algorithm Efficiency Chapter 2

CSC310 © Tom Briggs Shippensburg University Analysis Framework In what ways can we compare algorithms? In what ways can we compare algorithms? Time & Space efficiency are most common Time & Space efficiency are most common Others: Cost, Power, … Others: Cost, Power, … Characteristics of a good framework Characteristics of a good framework Independence from Independence from Programmer implementation Programmer implementation Language / compiler optimizations Language / compiler optimizations Architecture * Architecture * Dependence on Dependence on Input encoding Input encoding Critical operation of algorithm Critical operation of algorithm Frequency of critical operation execution Frequency of critical operation execution Number of “things” stored in memory relative to input encoding Number of “things” stored in memory relative to input encoding

CSC310 © Tom Briggs Shippensburg University Input Size Goal: Goal: express efficiency of an algorithm relative to the size of its encoded input. express efficiency of an algorithm relative to the size of its encoded input. We shall assume that all input to our algorithms is encoded efficiently. We shall assume that all input to our algorithms is encoded efficiently. Example, integers will typically be encoded in binary, which implies the following: Example, integers will typically be encoded in binary, which implies the following: Suppose we don’t make this assertion? Suppose we don’t make this assertion?

CSC310 © Tom Briggs Shippensburg University Measuring Running Time Basic Operation Basic Operation The operation contributing the most to total runtime The operation contributing the most to total runtime Frequency of execution depends on input Frequency of execution depends on input Let c _op be the execution time of an algorithms basic operation on a particular computer, and let C(n) be the number of times this operation is run on input n, then we can estimate the running time T(n) of a program: T(n) ¼ c op C(n)

CSC310 © Tom Briggs Shippensburg University Basic Operation What is the basic operation for: What is the basic operation for: Given some set, S = { i 1, i 2, … i n } of unordered integers, find the minimum Given some set, S = { i 1, i 2, … i n } of unordered integers, find the minimum Given two integers, a and b, find their sum, a + b Given two integers, a and b, find their sum, a + b Given a set, S = { i 1, i 2, … i n } and a, such that for all i k and a are all integers, determine whether S contains a. Given a set, S = { i 1, i 2, … i n } and a, such that for all i k and a are all integers, determine whether S contains a.

CSC310 © Tom Briggs Shippensburg University Unit Free Comparisons Goal: Goal: independence from architecture, implementation… independence from architecture, implementation… Suppose we observe that some algorithm, A, makes C(n) basic operations for an input of size n, where C(n) is: Suppose we observe that some algorithm, A, makes C(n) basic operations for an input of size n, where C(n) is: Suppose that each operation requires c 0 time T(n) ¼ c 0 C(n). How much longer would the algorithm require for 2 times its input i.e: T(2n) ¼ ? Remember the speed-up formula?

CSC310 © Tom Briggs Shippensburg University Orders of Growth

CSC310 © Tom Briggs Shippensburg University Big-Oh, Big-  O-notation: O-notation: A function t(n) is said to be in O(g(n)), denoted t 2 O(g(n)), if t(n) is bounded above by some constant multiple of g(n) for all large n. A function t(n) is said to be in O(g(n)), denoted t 2 O(g(n)), if t(n) is bounded above by some constant multiple of g(n) for all large n.  -notation:  -notation: A function t(n) is said to be in  (g(n)), denoted t 2 (  (n)), if t(n) is bounded below by some constant multiple of g(n) for all large n. A function t(n) is said to be in  (g(n)), denoted t 2 (  (n)), if t(n) is bounded below by some constant multiple of g(n) for all large n.

CSC310 © Tom Briggs Shippensburg University Big-  \Theta notation \Theta notation A function t(n) is said to be in \Theta(g(n)), denoted t(n) \in \Theta(g(n)), if t(n) is bounded both above and below by some positive constant multiples of g(n) for all large n, i.e. there must exists some positive constant c_1 and c_2 and some non- negative integer, n_0, such that: A function t(n) is said to be in \Theta(g(n)), denoted t(n) \in \Theta(g(n)), if t(n) is bounded both above and below by some positive constant multiples of g(n) for all large n, i.e. there must exists some positive constant c_1 and c_2 and some non- negative integer, n_0, such that:

CSC310 © Tom Briggs Shippensburg University Example of Big-Oh

CSC310 © Tom Briggs Shippensburg University Asymptotic Tightness Big-Oh does not have to be asymptotically tight Big-Oh does not have to be asymptotically tight t(n) = 0.5 n 2 O(n) with c = 1, n 0 = 1, but is also in O(n 100 )… t(n) = 0.5 n 2 O(n) with c = 1, n 0 = 1, but is also in O(n 100 )… Big-  isn’t tight either Big-  isn’t tight either t(n) = n 5 2  (n) with c=1, n 0 = 1… t(n) = n 5 2  (n) with c=1, n 0 = 1… Big-  is tight… Big-  is tight… t(n) must be in same growth classes to meet definition t(n) must be in same growth classes to meet definition Can you prove this assertion? Can you prove this assertion?

CSC310 © Tom Briggs Shippensburg University Prove Prove that t(n)=3n 3 +2n  (n 3 ). Show t(n) · c 1 g(n) for all n ¸ n 0 : i.e. t(n) 2 O(n 3 ) And, show c 0 g(n) · t(n) for all n ¸ n 0 : i.e. t(n) 2  (n 3 )

CSC310 © Tom Briggs Shippensburg University Theorem Theorem If t 1 (n) 2 O(g 1 (n)) and t 2 (n) 2 O(g 2 (n)), then t 1 (n)+t 2 (n) 2 O(max{g 1 (n), g 2 (n)}) Proof Sketch: By defn, we can take max(c 1 and c 2 ), and max(n 01, and n 02 )

CSC310 © Tom Briggs Shippensburg University General Plan for Non-Recursive Alg. 1. Decide on a parameter indicate input size 2. Identify the algorithm’s basic operation 3. Check whether the number of basic ops depends on input size 4. Set up a sum expressing the number of times the algorithm’s basic op is executed (see cheat sheet) 5. Use standard formulae to compute closed-form formula for the count, and compute its order of growth

CSC310 © Tom Briggs Shippensburg University Example Algorithm: UniqueElements // input A[0..n-1] // output true if all elements of A are unique // output true if all elements of A are unique for i à 0 to n-2 do for i à 0 to n-2 do for j à i +1 to n-1 do if A[i] = A[j] return false return true return true

CSC310 © Tom Briggs Shippensburg University Analysis of Recursive Algorithms Recall: Recursive algorithm – Solves smaller problem of same type Has a defined recursive case (calls self) Has a defined base case (trivial solution) All recursive cases eventually lead to a base case Frequently: no loops present (due to recursion) Goal: Find and solve recurrence relations that express run-time.

CSC310 © Tom Briggs Shippensburg University Example: What is the critical operation? What is the growth class?

CSC310 © Tom Briggs Shippensburg University Example (cont’d) What is the critical operation here? T(n)=? Backward substitution: start with some n, and work backward, substituting values for the C(n), until a clear pattern emerges; then substitute for the base case.

CSC310 © Tom Briggs Shippensburg University Another Example Tower of Hanoi Use backward substitution to solve this

CSC310 © Tom Briggs Shippensburg University Another Example What is the run-time if we assume that G is fully-connected, directed, weighted graph?