CS2420: Lecture 4 Vladimir Kulyukin Computer Science Department Utah State University

Outline Algorithm Analysis (Chapter 2)

Recall Big Question 1 I have designed an algorithm. How fast does it run?

Asymptotic Analysis We would like to determine time/space performance of programs regardless of constant factors (differences in compilers and operating systems and differences in programming languages). In asymptotic analysis, N is an instance characteristic, typically the size of the input to a program P that implements some algorithm A. F(N) is the time/space performance of P on instances of size N.

Model of Computation We have a standard digital computer. One instruction is executed at a time. There is a small finite set of primitive instructions that can represented an arbitrary complex instruction in any programming language. Every primitive instruction can be executed in constant time. We have sufficient memory to execute complex programs.

Computing The Arithmetic Sum int sum(int n) { 1int rslt = 0; 2for(int i = 1; i <= n; i++) 3rslt += i; 4return rslt; }

Computing The Arithmetic Sum Assignment on line 1 is executed once and takes 1 time unit. Assignment inside the for-loop is executed once and takes 1 time unit. Comparison inside the for-loop is executed n+1 times and takes (n+1) time units. Increment inside the for-loop is executed n times and takes n time units. Plus-equal on line 3 is executed n times and takes 2n time units. Return on line 4 is executed once and takes 1 time unit.

Computing The Arithmetic Sum We need to compute the sum total of the following time units: –1 time unit –n+1 time units –n time units –2n time units –1 time unit The sum total = 4 + 4n.

Computing The Arithmetic Sum What is the complexity of computing the arithmetic sum? T(N) = 4N + 4.

Computing The Arithmetic Sum int sum2(int n) { return n * (n + 1) / 2; }

Computing The Arithmetic Sum What is the complexity of sum? T(N) = 4.

Big-Oh Notation F(N) = O(G(N)) IFF there exist positive constants C>0 And N 0 > 0 such that F(N) ≤ CG(N) for All N ≥ N 0 The Objective is to find the Least (Smallest) Upper Bound

Big-Oh: Tight Bounds The objective is to find as tight an upper bound as possible. Loose bounds are easy but not that Useful F(N) = 4N is O(2 N ) is correct, but too loose to be useful. F(N) = 5N 2 Is O(N 4 ) is correct, but too loose to be useful.

Worst Case vs. Average Case Worst-Case Analysis (Big-Oh) gives an upper performance bound over all inputs of size N. This is the most useful and most frequent measure of performance. Average-Case Analysis gives an average time bound over all inputs of size N. Average case analysis is often hard to do, because it is unclear what is “average.”

Big-Oh of Polynomials Theorem: f(n) = a 0 + a 1 n + a 2 n 2 + … + a m n m, Then f(n) = O(n m ). Proof: f(n) ≤ ∑|a i |n i ≤ n m ∑|a i |n (i-m) ≤ n m ∑|a i |. Take C = ∑|a i |. Take n 0 = 1. f(n) = O(n m ).

Big-Oh of Polynomials: Example 1 Claim: F(N) = 3N + 2 = O(N). Since m = 1, then F(N) = O(N 1 ) = O(N).

Big-Oh of Polynomials: Example 2 Claim: F(N) = 5N 2 + 7N + 3 = O(N 2 ). Since m = 2, then F(N) = O(N 2 ).