1. 1.2. Comparing Algorithms

2. Learning outcomes Understand that algorithms can be compared by expressing their complexity as a function relative to the size of the problem. Understand that some algorithms are more efficient time-wise than other algorithms. Understand that some algorithms are more space-efficient than other algorithms. Understand the Order of Complexity: Linear time, polynomial time, exponential time.

3. What is an algorithm? Sequence of unambiguous instructions for solving a problem. For obtaining an output for any legitimate input in a finite amount of time. =Turing Machine programTuring Machine program

4. Why comparing algorithms? It is not good enough to just solve a problem; a problem needs to be solved efficiently Two measures:  Time  Memory Only becomes important with large amount of data or large problems

5. Complexity (space or time) Look at how time (or space) grows as size of problem input grows.=> Big O Notation Complexity is measured irrespectively of the machine speed Many programmers have had ugly surprises when they moved from small test data to large data sets. This analysis will make you aware of potential problems

6. Count 0 For Count 1 to n Sum Sum + Count End for Output Sum Natural Number123456 Running Total36101521 n=10 n=1 ∑ n=55 n=10 n=1 ∑ n=55

7. Mathematical Series Sum n = ½ n (n+1) Pseudocode : Sum n * (n + 1)/2 //compute sum of numbers Output Sum Sum=1/2 * 10 * (10+1) = 55 (reassuringly!) ∑

8. Activity Code both summation algorithms in VB.net and run to prove the answer is the same Declare the variable n as double and use VERY large numbers (100,000,000) and see if you can output the elapsed time for both methods. Count = 0 For Count = 1 to n Sum = Sum + Count End for Output Sum Sum = n * (n + 1)/2

9. Summary Always: an algorithm must be correct for ALL valid inputs!! But: the amount of memory used and speed of computation for a given set of values are also important criteria. Time efficiency depends on input size!

10. Order of Complexity Big-O (the "O" stands for "order of") notation is concerned with what happens for very large values of N, therefore only the largest term in a polynomial is needed. All smaller terms are dropped. Big-O describes how long it takes for an algorithm to be executed.

11. For example, the number of operations (comparisons) in some sorts is N 2 - N. (Example?) For large values of N, the single N term is insignificant compared to N 2, therefore one of these sorts would be described as an O(N 2 ) algorithm.

12. Here is a table of typical cases, showing how many "operations" would be performed for various values of N. Logarithms to base 2 (as used here) are proportional to logarithms in other base, so this doesn't affect the big-O formula.

13. constantlogarithmiclinear quadraticcubic nO(1)O(log N)O(N)O(N log N)O(N 2 )O(N 3 ) 1111111 2112248 412481664 81382464512 1614 642564,096 1,0241101,02410,240 1,048,57 6 1,073, 741,82 4 1,048,576120 1,048, 576 20,971,52010 12 10 16

14. Does anyone really have that much data? It's quite common. For example, it's hard to find a digital camera that has fewer than a million pixels (1 mega- pixel). These images are processed and displayed on the screen. The algorithms that do this had better not be O(N 2 )! If it took one microsecond (1 millionth of a second) to process each pixel, an O(N 2 ) algorithm would take more than a week to finish processing a 1 megapixel image, and more than three months to process a 3 megapixel image (note the rate of increase is definitely not linear).

15. Another example is sound. CD audio samples are 16 bits, sampled 44,100 times per second for each of two channels. A typical 3 minute song consists of about 8 million data points. You had better choose the right algorithm to process this data. A dictionary used for text analysis has about 125,000 entries. There's a big difference between a linear O(N), binary O(log N), or hash O(1) search.

16. Searching Type of SearchBig-OhComments Linear search array/ArrayList/LinkedList O(N) Binary search sorted array/ArrayList O(log N)Requires sorted data. Search balanced treeO(log N) Search hash tableO(1)

17. Sorting arrays/ArrayLists Type of Sort BestWorstAverageComments BubbleSortO(N)O(N 2 ) Not a good sort, except with ideal data. Insertion sort O(N)O(N 2 ) Perhaps best of O(N 2 ) sorts QuickSortO(N log N)O(N 2 )O(N log N)Good, but it worst case is O(N 2 )

18. Overview  Constant complexity O(1) – takes the same amount of time no matter how big the problem is.  Linear complexity O(n) – as the size of the problem increases, the time taken to solve the problem grows at the same rate.  Logarithmic complexity O(log n) – as the size of the problem grows, the time taken to solve the problem increases, but at a slower rate.

19.  Polynomial complexity O(n k ) – as the size of the problem grows, the time taken to solve it grows faster, but can be solved in a reasonable amount of time.  Exponential complexity O(k n ) – as the size of the problem grows, the time taken to solve it grows by larger and larger amounts. Only small size versions of these problems can be solved in a reasonable amount of time.  Factorial complexity O(n!) – even worse than exponential.