UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 11 Algorithm An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 12 Features of Algorithm “Besides merely being a finite set of rules that gives a sequence of operations for solving a specific type of problem, an algorithm has the five important features” [Knuth1] finiteness (otherwise: computational method) »termination definiteness »precise definition of each step input (zero or more) output (one or more) effectiveness »“Its operations must all be sufficiently basic that they can in principle be done exactly and in a finite length of time by someone using pencil and paper” [Knuth1]

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 13 Historical Perspective Muhammad ibn Musa al-Khwarizmi – 9 th century mathematician l l Euclid’s algorithm for finding the greatest common divisor

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 14 Greatest Common Divisor (GCD) Algorithm 1 Step 1 Assign the value of min{m,n} to t Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, to Step 4 Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4 Step 4 Decrease the value of t by 1. Go to Step 2 Note: m and n are positive integers

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 15 Correctness The algorithm terminates, because t is decreased by 1 each time we go through step 4, 1 divides any integer, and t eventually will reach 1 unless the algorithm stops earlier. The algorithm is partially correct, because when it returns t as the answer, t is the minimum value that divides both m and n

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 16 GCD Procedure 2 Step 1 Find the prime factors of m Step 2 Find the prime factors of n Step 3 Identify all the common factors in the two prime expansions found in Steps 1 and 2. If p is a common factor repeated i times in m and j times in n, assign to p the multiplicity min{i,j} Step 4 Compute the product of all the common factors with their multiplicities and return it as the GCD of m and n Note: as written, the algorithm requires than m and n be integers greater than 1, since 1 is not a prime

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 17 Procedure 2 or Algorithm 2? Procedure 2 is not an algorithm unless we can provide an effective way to find prime factors of a number The sieve of Eratosthenes is an algorithm that provides such an effective procedure

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 18 Euclid’s Algorithm E0 [Ensure m geq n] If m lt n, exchange m with n E1 [Find Remainder] Divide m by n and let r be the remainder (We will have 0 leq r lt n) E2 [Is it zero?] If r=0, the algorithm terminates; n is the answer E3 [Reduce] Set m to n, n to r, and go back to E1

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 19 Termination of Euclid’s Algorithm The second number of the pair gets smaller with each iteration and cannot become negative: indeed, the new value of n is r = m mod n, which is always smaller than n. Eventually, r becomes zero, and the algorithms stops.

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 110 Correctness of Euclid’s Algorithm After step E1, we have m = qn + r, for some integer q. If r = 0, then m is a multiple of n, and clearly in such a case n is the GCD of m and n. If r !=0, note that any number that divides both m and n must divide m – qn = r, and any number that divides both n and r must divide qn + r = m; so the set of divisors of {m,n} is the same as the set of divisors of {n,r}. In particular, the GCD of {m,n} is the same as the GCD of {n,r}. Therefore, E3 does not change the answer to the original problem. [Knuth1]

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 111 Algorithm Design Techniques Euclid’s algorithm is an example of a “Decrease and Conquer” algorithm: it works by replacing its instance by a simpler one (in step E3) It can be shown (exercise 5.6b) that the instance size, when measured by the second parameter, n, is always reduced by at least a factor of 2 after two successive iterations of Euclid’s algorithm In Euclid’s original, the remainder m mod n is computed by repeated subtraction of n from m Step E0 is not necessary for correctness; it improves time efficiency

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 112 Notion of algorithm “computer” Algorithmic solution problem algorithm inputoutput

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 113 Example of computational problem: sorting Statement of problem: –Input: A sequence of n numbers –Output: A reordering of the input sequence so that a ´ i ≤ a ´ j whenever i < j Instance: The sequence Algorithms: –Selection sort –Insertion sort –Merge sort –(many others)

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 114 Selection Sort Input: array a[1],…,a[n] Output: array a sorted in non-decreasing order Algorithm: for i =1 to n swap a[i] with smallest of a[i+1],…a[n] see also pseudocode, section 3.1

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 115 Sorting: Some Terms Key Stable sorting In place sorting

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 116 Graphs: Some Terms Graph Vertices, edges Undirected, directed Loops Complete, dense, sparse Paths: sequences of vertices or of edges? Path length: number of edges in path Walk: repeated vertices and edges allowed Path: repeated vertices are allowed, repeated edges are not allowed Simple path: neither repeated vertices nor repeated edges are allowed Cycle: simple path that starts and ends at the same vertex

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 117 Some Well-known Computational Problems Sorting Searching Shortest paths in a graph Minimum spanning tree Primality testing Traveling salesman problem Knapsack problem Chess Towers of Hanoi Program termination

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 118 Basic Issues Related to Algorithms How to design algorithms How to express algorithms Proving correctness Efficiency –Theoretical analysis –Empirical analysis Optimality

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 119 Algorithm design strategies Brute force Divide and conquer Decrease and conquer Transform and conquer Greedy approach Dynamic programming Backtracking and Branch and bound Space and time tradeoffs See This is not a unique taxonomy.

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 120 Analysis of Algorithms How good is the algorithm? –Correctness –Time efficiency –Space efficiency Does there exist a better algorithm? –Lower bounds –Optimality

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 121 What is an algorithm? Recipe, process, method, technique, procedure, routine,… with the following requirements: 1.Finiteness bterminates after a finite number of steps 2.Definiteness brigorously and unambiguously specified 3.Input bvalid inputs are clearly specified 4.Output bcan be proved to produce the correct output given a valid input 5.Effectiveness bsteps are sufficiently simple and basic

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 122 Why study algorithms? Theoretical importance –the core of computer science Practical importance –A practitioner’s toolkit of known algorithms –Framework for designing and analyzing algorithms for new problems

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 123 Correctness Termination Well-founded sets: find a quantity that is never negative and that always decreases as the algorithm is executed Partial Correctness For recursive algorithms: induction For iterative algorithms: axiomatic semantics, loop invariants

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Design and Analysis of Algorithms - Chapter 124 Complexity Space complexity Time complexity For iterative algorithms: sums For recursive algorithms: recurrence relations