1. Design and Analysis of Algorithms - Chapter 1
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
Design and Analysis of Algorithms - Chapter 1

2. Design and Analysis of Algorithms - Chapter 1
Features of Algorithm
“Besides merely being a finite set of rules that gives a sequence of operations for solving a specific type of problem, an algorithm has the five important features” [Knuth1]
finiteness (otherwise: computational method)
termination
definiteness
precise definition of each step
input (zero or more)
output (one or more)
effectiveness
“Its operations must all be sufficiently basic that they can in principle be done exactly and in a finite length of time by someone using pencil and paper” [Knuth1]
Design and Analysis of Algorithms - Chapter 1

3. Historical Perspective
Muhammad ibn Musa al-Khwarizmi – 9th century mathematician
Euclid’s algorithm for finding the greatest common divisor
Euclid’s algorithm is good for introducing the notion of an algorithm because it
makes a clear separation from a program that implements the algorithm.
It is also one that is familiar to most students.
Al Khowarizmi (many spellings possible...) – “algorism” (originally) and then
later “algorithm” come from his name.
Design and Analysis of Algorithms - Chapter 1

4. Greatest Common Divisor (GCD) Algorithm 1
Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, to Step 4
Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4
Step 4 Decrease the value of t by 1. Go to Step 2
Note: m and n are positive integers
Design and Analysis of Algorithms - Chapter 1

5. Design and Analysis of Algorithms - Chapter 1
Correctness
The algorithm terminates, because t is decreased by 1 each time we go through step 4, 1 divides any integer, and t eventually will reach 1 unless the algorithm stops earlier.
The algorithm is partially correct, because when it returns t as the answer, t is the minimum value that divides both m and n
Design and Analysis of Algorithms - Chapter 1

6. Design and Analysis of Algorithms - Chapter 1
GCD Procedure 2
Step 1 Find the prime factors of m
Step 2 Find the prime factors of n
Step 3 Identify all the common factors in the two prime expansions found in Steps 1 and 2. If p is a common factor repeated i times in m and j times in n, assign to p the multiplicity min{i,j}
Step 4 Compute the product of all the common factors with their multiplicities and return it as the GCD of m and n
Note: as written, the algorithm requires than m and n be integers greater than 1, since 1 is not a prime
Design and Analysis of Algorithms - Chapter 1

7. Procedure 2 or Algorithm 2?
Procedure 2 is not an algorithm unless we can provide an effective way to find prime factors of a number
The sieve of Eratosthenes is an algorithm that provides such an effective procedure
Design and Analysis of Algorithms - Chapter 1

8. Design and Analysis of Algorithms - Chapter 1
Euclid’s Algorithm
E0 [Ensure m geq n] If m lt n, exchange m with n
E1 [Find Remainder] Divide m by n and let r be the remainder (We will have 0 leq r lt n)
E2 [Is it zero?] If r=0, the algorithm terminates; n is the answer
E3 [Reduce] Set m to n, n to r, and go back to E1
Design and Analysis of Algorithms - Chapter 1

9. Termination of Euclid’s Algorithm
The second number of the pair gets smaller with each iteration and cannot become negative: indeed, the new value of n is r = m mod n, which is always smaller than n. Eventually, r becomes zero, and the algorithms stops.
Design and Analysis of Algorithms - Chapter 1

10. Correctness of Euclid’s Algorithm
After step E1, we have m = qn + r, for some integer q. If r = 0, then m is a multiple of n, and clearly in such a case n is the GCD of m and n. If r !=0, note that any number that divides both m and n must divide m – qn = r, and any number that divides both n and r must divide qn + r = m; so the set of divisors of {m,n} is the same as the set of divisors of {n,r}. In particular, the GCD of {m,n} is the same as the GCD of {n,r}. Therefore, E3 does not change the answer to the original problem. [Knuth1]
Design and Analysis of Algorithms - Chapter 1

11. Design and Analysis of Algorithms - Chapter 1
Book Variation (p.4)
Step 1 If n=0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step 2 Divide m by n and assign the value of the remainder to r.
Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.
Note: m and n are nonnegative, not-both-zero integers
Design and Analysis of Algorithms - Chapter 1

12. Pseudocode for book version
while n <> 0 do
r <- m mod n
m <- n
n <- r
return m
Design and Analysis of Algorithms - Chapter 1

13. Algorithm Design Techniques
Euclid’s algorithm is an example of a “Decrease and Conquer” algorithm: it works by replacing its instance by a simpler one (in step E3)
It can be shown (exercise 5.6b) that the instance size, when measured by the second parameter, n, is always reduced by at least a factor of 2 after two successive iterations of Euclid’s algorithm
In Euclid’s original, the remainder m mod n is computed by repeated subtraction of n from m
Step E0 is not necessary for correctness; it improves time efficiency
Design and Analysis of Algorithms - Chapter 1

14. Design and Analysis of Algorithms - Chapter 1
Notion of algorithm
problem
algorithm
“computer”
input
output
Algorithmic solution
Design and Analysis of Algorithms - Chapter 1

15. Example of computational problem: sorting
Statement of problem:
Input: A sequence of n numbers <a1, a2, …, an>
Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i ≤ a´j whenever i < j
Instance: The sequence <5, 3, 2, 8, 3>
Algorithms:
Selection sort
Insertion sort
Merge sort
(many others)
Design and Analysis of Algorithms - Chapter 1

16. Selection Sort Input: array a[1],…,a[n]
Output: array a sorted in non-decreasing order
Algorithm:
for i=1 to n
swap a[i] with smallest of a[i+1],…a[n]
The algorithm is given *very* informally here. Show students the pseudocode in
section 3.1.
This is a good opportunity to discuss pseudocode conventions.
see also pseudocode, section 3.1
Design and Analysis of Algorithms - Chapter 1

17. Design and Analysis of Algorithms - Chapter 1
Sorting: Some Terms
Key
Stable sorting
In place sorting
Design and Analysis of Algorithms - Chapter 1

18. Design and Analysis of Algorithms - Chapter 1
Graphs: Some Terms
Graph
Vertices, edges
Undirected, directed
Loops
Complete, dense, sparse
Paths: sequences of vertices or of edges?
Path length: number of edges in path
Walk: repeated vertices and edges allowed
Path: repeated vertices are allowed, repeated edges are not allowed
Simple path: neither repeated vertices nor repeated edges are allowed
Cycle: simple path that starts and ends at the same vertex
Design and Analysis of Algorithms - Chapter 1

19. Some Well-known Computational Problems
Sorting
Searching
Shortest paths in a graph
Minimum spanning tree
Primality testing
Traveling salesman problem
Knapsack problem
Chess
Towers of Hanoi
Program termination
1-4 have well known efficient (polynomial-time) solutions
5: primality testing has recently been found to have an efficient solution
This is a great problem to discuss because it has recently been in the news
(see mathworld news at:
or original article:
6(TSP)-9(chess) are all problems for which no efficient solution has been found
it is possible to informally discuss the “try all possibilities” approach that is required
to get exact solutions to such problems
10: Towers of Hanoi is a problem that has only exponential-time solutions (simply
because the output required is so large)
11: Program termination is undecidable
Design and Analysis of Algorithms - Chapter 1

20. Basic Issues Related to Algorithms
How to design algorithms
How to express algorithms
Proving correctness
Efficiency
Theoretical analysis
Empirical analysis
Optimality
Design and Analysis of Algorithms - Chapter 1

21. Algorithm design strategies
See
This is not a unique taxonomy.
Greedy approach
Dynamic programming
Backtracking and Branch and bound
Space and time tradeoffs
Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Design and Analysis of Algorithms - Chapter 1

22. Analysis of Algorithms
How good is the algorithm?
Correctness
Time efficiency
Space efficiency
Does there exist a better algorithm?
Lower bounds
Optimality
Design and Analysis of Algorithms - Chapter 1

23. Design and Analysis of Algorithms - Chapter 1
What is an algorithm?
Recipe, process, method, technique, procedure, routine,… with the following requirements:
Finiteness
terminates after a finite number of steps
Definiteness
rigorously and unambiguously specified
Input
valid inputs are clearly specified
Output
can be proved to produce the correct output given a valid input
Effectiveness
steps are sufficiently simple and basic
The formalization of the notion of an algorithm led to great breakthroughs in the
foundations of mathematics in the 1930s.
Design and Analysis of Algorithms - Chapter 1

24. Design and Analysis of Algorithms - Chapter 1
Why study algorithms?
Theoretical importance
the core of computer science
Practical importance
A practitioner’s toolkit of known algorithms
Framework for designing and analyzing algorithms for new problems
Design and Analysis of Algorithms - Chapter 1

25. Design and Analysis of Algorithms - Chapter 1
Correctness
Termination
Well-founded sets: find a quantity that is never negative and that always decreases as the algorithm is executed
Partial Correctness
For recursive algorithms: induction
For iterative algorithms: axiomatic semantics, loop invariants
Design and Analysis of Algorithms - Chapter 1

26. Design and Analysis of Algorithms - Chapter 1
Proving Correctness
Make sure that the algorithm will terminate.
Define a precondition (allowable input and state) and a postcondition (correct output).
Show that, given the precondition, algorithm will fulfill the postcondition.
In many cases, the algorithm is either recursive or iterative. Determine which one applies.
Design and Analysis of Algorithms - Chapter 1

27. Design and Analysis of Algorithms - Chapter 1
Recursive Algorithms
Use induction and show correct fulfillment of postcondition in:
basis case (input is of size 0)
case where input is of size n+1, assuming (inductive assumption) that postcondition is fulfilled when input is of size n (simple induction) or input is of all sizes from 0 to n (strong or complete induction)
Design and Analysis of Algorithms - Chapter 1

28. Design and Analysis of Algorithms - Chapter 1
Iterative Algorithms
Determine an appropriate invariant.
Show that the invariant is true before entering the loop for the first time.
Show that, if the invariant is true and the loop condition is false, the postcondition is fulfilled.
Show, by induction, that, if the invariant is true for input of size n and the loop condition is true, the invariant is also true for input of size n+1.
Design and Analysis of Algorithms - Chapter 1

29. Example: Recursive Factorial
(* Pre: i GEQ 0 *)
fun fact(i) = if i = 0 then 1 else i * fact(i-1);
(* Post: fact(i) = i! *)
Design and Analysis of Algorithms - Chapter 1

30. Example: Iterative Factorial
(* Pre’: i>= 0*)
fun fact(i)
int j=1, f=1;
(* Pre: i>=0 and j=f=1 *)
while j <= i
f = f * j;
j := j+1;
(* Post: f = i! *)
return f;
(* Post’: fact(i) = i! *)
Design and Analysis of Algorithms - Chapter 1

31. Example: Product Algorithm
To be added
Design and Analysis of Algorithms - Chapter 1

32. Example: Recursive BinarySearch
To be added
Design and Analysis of Algorithms - Chapter 1

33. Example: Iterative Binary Search
To be added
Design and Analysis of Algorithms - Chapter 1

34. Example: Recursively Computing Binomial Coefficients
Multiple calls
To be added
Design and Analysis of Algorithms - Chapter 1

35. Example: RecursiveSelectionSort
(* Author: Kingston *) (* Pre: a LTE b + 1 *) (* SelectionSort(A, a, b) *) (* Post: A[a] LTE A[a+1] LTE A[a+2] LTE ... A[b] *) (* and A is a permutation of the input array *) procedure SelectionSort (var A: AType; a,b: integer); var i: integer; begin if a = b + 1 then (* do nothing *) else i := MinIndex(A, a, b); if (i <> a) then Swap(A[i],A[a]); end; SelectionSort(A, a + 1, b); end; end SelectionSort; (* MinIndex returns the index of a minimum element of the non-empty array A[i..j], and Swap(A[i],A[j] swaps A[i] and A[j] *)
Design and Analysis of Algorithms - Chapter 1

36. Example: Max Contiguous Subvector
Algorithm: Maximum-Sum Contiguous Subvector
/* Precondition: x[1..n] is an array of floating-point (real) numbers. */
procedure MaxSumContigVector (x[1..n]: float) // Returns the maximum sum of a continuous subarray drawn from x. MaxSoFar := 0.0 MaxEndingHere := 0 i := 1 while i ≠ n + 1 do MaxEndingHere := max(0.0, MaxEndingHere + x[i]) MaxSoFar := max(MaxSoFar, MaxEndingHere) i := i + 1
end return MaxSoFar
/* Postcondition: MaxSoFar is the maximum sum of any contiguous subvector in x[1..n]. */
Design and Analysis of Algorithms - Chapter 1

37. Design and Analysis of Algorithms - Chapter 1
Complexity
Space complexity
Time complexity
For iterative algorithms: sums
For recursive algorithms: recurrence relations
Design and Analysis of Algorithms - Chapter 1