CSE 3358 NOTE SET 2 Data Structures and Algorithms 1

Overview: 2  What are we measuring and why?  Computational complexity introduction  Big-O Notation

Problem 3  For a problem  Different ways to solve – differing algorithms  Problem: Searching an element in an array Possible searching algorithms? 

The Selection Problem 4  Problem: Find the Kth largest number in a set  Solution Possibilities: 

The Selection Problem 5  If dataset size = 10,000,000 and k = 5,000,000  Previous algos could take several days  Is one algorithm better than the other?  Are there better algorithms?

Efficiency 6  Limited amount of resources to use in solving a problem.  Resource Examples:  We can use any metric to compare various algorithms intended to solve the same problem.  Using some metrics in for some problems might not be enlightening…

Computational Complexity 7  Computational complexity – the amount of effort needed to apply an algorithm or how costly it is.  Two most common metrics (and the ones we’ll use)  Time (most common)  Space  Time to execute an algorithm on a particular data set is system dependent. Why?

Seconds? Microseconds? Nanoseconds? 8  Can’t use the above when talking about algorithms unless specific to a particular machine at a particular time.  Why not?

What will we use? 9  Logical units that express the relationship between the size n of a data set and the amount of time t required to process the data  For algorithm X using dataset n, T(n) = amount of time needed to execute X using n.  Problem: Ordering of the values in n can affect T(n). What to do?

How we measure resource usage 10  Three primary ways of mathematically discussing the amount of resources used by an algorithm  O(f(n))  Ω (f(n))   (f(n))  What is ___(f(n)) ? 

The Definitions 11  T(N) = O(f(n)) if there are positive constants c and n 0 such that T(N) = n 0.  T(N) = Ω (g(n)) if there are positive constants c and n 0 such that T(N) >= c*g(n) when N >= n 0.  T(N) =  (h(n)) iff T(N) = O(h(n)) and T(N) = Ω (h(n))

The Goal??? 12  To place a relative ordering on functions  To examine the relative rates of growth.  Example:

Big - Oh 13  T(N) = O(f(n))  What does this really mean?  Means:  T is big-O of f if there is a positive number c such that T is not larger than c*f for sufficiently large ns (for all ns larger than some number N)  In other words:  The relationship between T and f can be expressed by stating either that f(n) is an upper bound on the value of T(n) or that, in the long run, T grows at most as fast as f.

Asymptotic Notation: Big-O 14 Graphic Example

Asymptotic Notation: Big-Oh 15 Example: Show that 2n 2 + 3n + 1 is O(n 2 ). By the definition of Big-O 2n 2 + 3n + 1 = N. So we must find a c and N such that the inequality holds for all n > N. How?

Asymptotic Notation: Big-O 16  Reality Check:  We’re interested in what happens to the number of ops needed to solve a problem as the size of the input increases toward infinity.  Not too interested in what happens with small data sets.

Notes on Notation 17  Very common to see T(n) = O(f(n))  Not completely accurate  not symmetric about =  Technically, O(f(n)) is a set of functions.  Set definition O(g(n)) = { f(n): there are constants c > 0, N>0 such that 0 N.}  When we say f(n) = O(g(n)), we really mean that f(n) ∈ O(g(n)).

Asymptotic Notation: Big-O 18 Show that n 2 = O(n 3 ).

Three Cases for Analysis 19  Best Case Analysis:  when the number of steps needed to complete an algorithm with a data set is minimized  e.g. Sorting a sorted list  Worst Case Analysis:  when the maximum number of steps possible with an algorithm is needed to solve a problem for a particular data set  Average Case Analysis:

Example 20  The problem is SORTING (ascending)  Best Case:  Worst Case:  Average Case: Data Set: n = _______

T(N)? 21  For a particular algorithm, how do we determine T(N)?  Use a basic model of computation. Instructions are executed sequentially Standard simple instructions Add, subtract, multiply, divide Comparison, store, retrieve  Assumptions: Takes one time unit, T(1) to do anything simple

Determining Complexity  How can we determine the complexity of a particular algorithm? int sum(int* arr, int size) { int sum = 0; for (int i = 0; i < size; i++) sum += arr[i]; return sum; }

23 ?