ALGORITHMS Introduction

Definition Algorithm: Any well-defined computational procedure that takes some value or set of values as input and produces some value or set of values as output.

Properties of Algorithms Input – an algorithm has input values from a specified set Output – for each set of input values, an algorithm produces output values from a specified set Definiteness – the steps of the algorithm must be defined precisely

Properties of Algorithms (cont) Finiteness – an algorithm should produce the desired output after a finite number of steps for any input in the set Effectiveness – it must be possible to perform each step of an algorithm exactly in a finite amount of time Generality – the algorithm should be applicable for all problems of the desired form, not just a set of input values

Question 1 How do you show that an algorithm is incorrect?

Question 2 How do you show that an algorithm is correct ?

Computational Model for Analysis One processor random-access machine (RAM) – instructions are executed one at a time – no concurrent operations

Choosing the “Best” Algorithm Empirical approach – try competing algorithms on several different problem instances Theoretical approach – determine mathematically the quantity of resources (execution time and memory) needed by the algorithm as a function of the size of the instance considered

What is “Input Size”? Problem dependent Identification of objects in problem processed by algorithm – Number of items in the input is often used (e.g. length of an array) – Sometimes use number of bits – May use more than one measure (number of edges and vertices in a graph)

What is the “Running Time” Number of primitive operations or “steps” executed. Want this definition to be machine independent Initial view taken by text – constant amount of time for each line of pseudocode

Insertion Sort Example Commonly used sorting algorithm Sorting of playing cards often used as an analogy

Initially sorted list key 2 A

key 2 A

A

key 4 A

key 4 A

A

INSERTION-SORT (A)costtime 1for j  2 to length[A] doc 1 n 2key  A[j]c 2 n-1 3;Insert A[j] into the sorted ; sequence A[1.. j -1]0n-1 4i  j - 1c 4 n-1 5while i > 0 and A[i] > key doc 5 6 A[i+1]  A[i]c 6 7 i  i- 1c 7 8A[i+1]  keyc 8 n-1

Running Time for Insertion Sort

Why Worst Case Analysis? Gives an upper bound on the running time for any input For some algorithms, the worst case occurs fairly often Difficult to identify the average case Difficult to analyze the average case The average case is often roughly as bad as the worst case.

Order of Growth Consider only the highest order terms in formulas Ignore the leading term’s constant coefficient For insertion sort the worst case running time is  (n 2 )

Designing Algorithms A number of design paradigms for algorithms that have proven useful for many types of problems Insertion sort – incremental approach Other examples of design approaches – divide and conquer – greedy – dynamic programming

Divide and Conquer Good divide and conquer algorithm generally implies easy recursive version of the algorithm Three steps – divide – conquer – combine

Merge Sort Divide divide and n-element sequence into two n/2 element sequences Conquer if the resulting list is of length 1 it is sorted else call the merge sort recursively Combine merge the two sorted sequences

MERGE-SORT (A,p,r) 1if p < r 2then q  (p+r)/2  3MERGE-SORT(A,p,q) 4MERGE-SORT(A,q+1,r) 5MERGE(A,p,q,r) To sort A[1..n], invoke MERGE-SORT with MERGE-SORT(A,1,length(A))

Merge 1 3 Merge Merge Merge 2 5 Merge 2 6 Merge initial sequence sorted sequence Merge

Recurrence for Divide and Conquer Algorithms

Recurrence for Merge Sort