Algorithms Friday 7th Week

Algorithms What is an Algorithm? –A series of precise steps, known to stop eventually, to solve a problem –NOT necessarily tied to computers –There can be many algorithms to solve the same problem

Characteristics of an Algorithm 1.Precise steps 2.Effective steps 3.Has an output 4.Terminates eventually

Trivial Algorithm Computing an average: 1.Add up all of the values 2.Divide by the number of values

Greatest Common Divisor Computing the Greatest Common Divisor (gcd) of two numbers Examples: –gcd(12, 2) = 2 –gcd(100, 95) = 5 –gcd(100, 75) = 25 –gcd(39, 26) = 13 –gcd(17, 8) = 1

Possible Algorithm #1 Assumption: A > B >= 0 –If A is a multiple of B, then gcd(A, B) = B. –Otherwise, return an error. Works for gcd(12,2) = 2 But what about gcd(100, 95)???

Possible Algorithm #2 –Start with 1 and go up to B. –If a number if a common divisor of both A and B, remember it. –When we get to B, stop. The last number we remembered is the gcd. Works, but is there a better way? Think about gcd(100, 95)

Euclid’s Algorithm Make use of the fact that: gcd(A, B) = gcd(B, A rem B) –Note: A rem B refers to the remainder left when A is divided by B. –Examples: 12 rem 2 = rem 95 = 5

Euclid’s Algorithm –If B = 0, then gcd(A, B) = A –Otherwise, gcd(A, B) = gcd (B, A rem B) Note – this algorithm is recursive. Examples: –gcd(12, 2) = gcd(2, 0) = 2 –gcd(100, 95) = gcd(95, 5) = gcd(5, 0) = 5

Program vs. Algorithm Program: “A set of computer instructions written in a programming language” We write Programs (in JavaScript) that implement various Algorithms

Program vs. Algorithm Problem: Sort a list of numbers Algorithms: –Quicksort –Insertion Sort –Merge Sort Programs: –Using JavaScript to implement Insertion Sort –Using Pascal to implement Quicksort

Algorithms to Solve Problems Problems can be categorized into two categories –Computable problems can be solved using an algorithm. –Non-computable problems have no algorithm to solve them. We don’t necessariliy know which category a problem is in. (this is non- computable)

Computable Problems Tractable –There is an efficient algorithm to solve the problem. Intractable –There is an algorithm to solve the problem but the best known algorithm is not efficient.

Examples Sorting: tractable Chess Playing: intractable Halting Problem: non-computable

How do we measure efficiency? To do this, we count the number of steps an algorithm takes. For example, consider the following algorithm which adds up the numbers in an array:

Add up the numbers in an array sum = 0; for (var i=0; i<numArray.length; i++) { sum += numArray[i]; } The statement inside the for loop gets executed numArray.length times – it depends on what the length is. If the length is n, we say this algorithm is “on the order of n”, or, this algorithm is O(n).

Definition of O(g(n)) f(n)=O(g(n)) means that for all n larger than some threshold, f(n) <= C*g(n) for some constant C, Or, the ratio f(n)/g(n) <= C. This means that f grows no faster than g, or f is proportional to g. CONSTANTS DON’T MATTER f could represent time an algorithm takes, or the number of steps executed.

Big O Analysis What is the worst case running time on input of size N, discounting constants

Comparison of Functions Look at graphs of f(x)=x, g(x)=x 2, h(x)=x 3, j(x)=logx k(x) = 2 x How do they grow? How do they compare with each other?

Tractable vs. Intractable Polynomial time algorithms are considered tractable. Exponential time algorithms are considered intractable.

Sorting (Insertion Sort) function insertionSort (numArray) { for ( var i=1; i < numArray.length; i++) { var temp = numArray[i]; for (var j = i; (j > 0) && ( numArray[j-1] > temp); j--) { numArray[j] = numArray[j-1]; } numArray[j] = temp; }

Sorting (Merge Sort) Given a list, split it into 2 equal piles. Then split each of these piles in half. Continue to do this until you are left with 1 element in each pile. Now merge piles back together in order.

(Merge Sort, cont) An example of how the merge works: Suppose the first half and second half of an array are sorted: Merge these by taking a new element from either the first or second subarray, choosing the smallest of the remaining elements.

Big O Can Be Deceiving Big O analysis is concerned with worst case behavior Sometimes we know that we are not dealing with the worst case

Search the Array for (var i=0; i<numArray.length; i++) { if (numArray[i] == “what we want”) stop and jump for joy; }

Binary Search If we are searching in a sorted list, we choose the half that the value would be in. We continue to cut the area we are searching in half until we find the value we want or there are no more values to check.

Algorithms Exercise