0. Chapter 1
Introduction
Levitin: Introduction to The Design and Analysis of Algorithms

1. Why study algorithms? Theoretical importance
the core of computer science
Practical importance
A practitioner’s toolkit of known algorithms
Framework for designing and analyzing algorithms for new problems

2. Two main issues related to algorithms
How to design algorithms
How to analyze algorithm efficiency

3. What is an algorithm?
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
problem
algorithm
An algorithm is a well-ordered collection of unambiguous and effectively computable operations that, when executed, produces a result and halts in a finite amount of time.
“computer”
input
output

4. What is an algorithm?
Recipe, process, method, technique, procedure, routine,… with following requirements:
Finiteness
terminates after a finite number of steps
Definiteness
rigorously and unambiguously specified
Input
valid inputs are clearly specified
Output
can be proved to produce the correct output given a valid input
Effectiveness
steps are sufficiently simple and basic
The formalization of the notion of an algorithm led to great breakthroughs in the
foundations of mathematics in the 1930s.

5. Alternative definition
An algorithm is a well-ordered collection of unambiguous and effectively computable operations that, when executed, produces a result and halts in a finite amount of time.
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
Muhammad ibn Musa al-Khwarizmi – 9th century mathematician

6. Historical Perspective
Euclid’s algorithm for finding the greatest common divisor
third century BC
What are alternative ways we can compute the gcd of two integers?
Euclid’s algorithm is good for introducing the notion of an algorithm because it
makes a clear separation from a program that implements the algorithm.
It is also one that is familiar to most students.
Al Khowarizmi (many spellings possible...) – “algorism” (originally) and then
later “algorithm” come from his name.

7. Euclid’s Algorithm
Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n
Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?
Euclid’s algorithm is based on repeated application of equality
gcd(m,n) = gcd(n, m mod n)
until the second number becomes 0, which makes the problem
trivial.
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
Euclid’s algorithm is good for introducing the notion of an algorithm because it
makes a clear separation from a program that implements the algorithm.
It is also one that is familiar to most students.
Al Khowarizmi (many spellings possible...) – “algorism” (originally) and then
later “algorithm” come from his name.

8. Two descriptions of Euclid’s algorithm
Step 1 If n = 0, return m and stop; otherwise go to Step 2
Step 2 Divide m by n and assign the value fo the remainder to r
Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.
while n ≠ 0 do
r ← m mod n
m← n
n ← r
return m

9. Other methods for computing gcd(m,n)
Consecutive integer checking algorithm
Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4
Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4
Step 4 Decrease t by 1 and go to Step 2

10. Other methods for gcd(m,n) [cont.]
Middle-school procedure
Step 1 Find the prime factorization of m
Step 2 Find the prime factorization of n
Step 3 Find all the common prime factors
Step 4 Compute the product of all the common prime factors and return it as gcd(m,n)
Is this an algorithm?

11. Sieve of Eratosthenes Input: Integer n ≥ 2
Output: List of primes less than or equal to n
for p ← 2 to n do A[p] ← p
for p ← 2 to n do
if A[p]  0 //p hasn’t been previously eliminated from the list j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j ← j + p
Example:

12. Basic Issues Related to Algorithms
How to design algorithms
How to express algorithms
Proving correctness
Efficiency
Theoretical analysis
Empirical analysis
Optimality

13. 1.2 Fundamentals of Algorithmic Problem Solving
Algorithms are procedural solutions to problems
Steps for designing and analyzing an algorithm
Understand the problem
Ascertain the capabilities of a computational device
Choose between exact and approximate problem solving
Decide on appropriate data structures

14. Fundamentals of Algorithmic Problem Solving, continued: Algorithm design techniques/strategies
Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Space and time tradeoffs
Greedy approach
Dynamic programming
Iterative improvement
Backtracking
Branch and bound

15. Fundamentals of Algorithmic Problem Solving continued
Methods of specifying an algorithm
Present: pseudocode
Earlier: flowchart
Proving an algorithm’s correctness
Mathematical induction for recursion
Modus ponens
E.g. algorithm stops since number gets smaller on each iteration
Approximation algorithms are more difficult

16. How good is the algorithm?
Fundamentals of Algorithmic Problem Solving, continued: Analysis of Algorithms and Coding
How good is the algorithm?
Correctness
Time efficiency
Space efficiency
Does there exist a better algorithm?
Lower bounds
Optimality
Algorithm ultimately implemented as a computer program
Success depends on above considerations

17. 1.3 Important problem types
sorting
searching
string processing
graph problems
combinatorial problems
geometric problems
numerical problems

18. Example of computational problem: sorting
Statement of problem:
Input: A sequence of n numbers <a1, a2, …, an>
Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i ≤ a´j whenever i < j
Instance: The sequence <5, 3, 2, 8, 3>
Algorithms:
Selection sort
Insertion sort
Merge sort
(many others)

19. see also pseudocode, section 3.1
Selection Sort
Input: array a[1],…,a[n]
Output: array a sorted in non-decreasing order
Algorithm:
for i=1 to n
swap a[i] with smallest of a[i],…a[n]
The algorithm is given *very* informally here. Show students the pseudocode in
section 3.1.
This is a good opportunity to discuss pseudocode conventions.
see also pseudocode, section 3.1

20. Fundamental data structures
graph
tree
set and dictionary
list
array
linked list
string
stack
queue
priority queue

21. Data Structure Definition:
“A particular scheme of organizing related data items.”
Example:
Array
String
List
Classified into
Linear
Graphs
Tree
Abstract

22. Linear Structures Array String (Type of Mathematical Sequence)
Character, Binary, Bit…
Linked List
Nodes, Head, Tail
Singly Linked, Doubly Linked,
Stack
LIFO, Top, Push, Pop
Queue
FIFO, Front, Rear/End, Enqueue, Dequeue

23. Graphs
Graph –
Def: “A graph G = <V,E> is defined by a pair of two sets: a finite set V of items called vertices and a set E of pairs (2-tuples) of vertices that represent the edges between the vertices.
Undirected, Directed (digraph)
Loops
Complete, Sparse, Dense

24. { (A,B),(A,C),(B,A),(B,B),(C,C) }
Representing Graphs
Graphs –
Inherently abstract
Directed or Undirected
Two common representations:
Adjacency matrix int P[][] = new int P[][];
Adjacency linked list
LinkedList<Edge> Q = new LinkedList<Edge>( ); A B C 1
{ (A,B),(A,C),(B,A),(B,B),(C,C) }

25. Weighted Graphs Weighted Graph –
A graph G, with the added property that the edges between nodes is assigned a “weight”
Edge weight may represent: cost, distance, time… A B C
1.5
2.3
1.3
2.0
1.1
{ (A,B,1.5),(A,C,2.3),(B,A,1.3), (B,B,2.0),(C,C,1.1) }

26. Graph Terms (1/2) Path – Simple – Length of a path –
A path from vertex u and v of a graph G exists if there exists a set adjacent edges that starts with u and ends with v
Simple –
The path contains only unique edges (no duplicates)
Length of a path –
The number of edges in the aforementioned set

27. Graph Terms (2/2) Connected –
iff, for every pair of its vertices, <u,v>, there is a path from u to v.
Connected Components –
If a graph is not connected, it is made of two or more connected components.
The “maximal” connected subgraph of a given graph.
Cycle –
A simple path, of non-zero length, that starts & ends at the same vertex.
Acyclic Graph –
A graph with no cycles

28. Trees Tree Forest A connected acyclic graph Properties A set of trees
|E| = |V|-1
For every two vertices, <u,v>, there exists exactly one simple path from u to v.
Forest
A set of trees
An acyclic graph that is not necessarily connected

29. Rooted Trees Rooted Tree Tree genealogy A tree, with a “root” node
Root nodes are selected arbitrarily (any vertex)
Tree genealogy
Ancestors
Children
Siblings
Parents
Descendents
Leaf nodes

30. Example D G A B H E C F Convert the graph to a rooted tree
Identify the root node
Find an ancestor, child, sibling, parent, descendent, and leaf nodes D G A B H E C F

31. Height & Depth of Trees Depth Height
Depth of vertex v is the length of the simple path from the root to v
Count the edges
Height
Height of a tree is the length of the longest simple path from the root to the deepest leaf nodes

32. Binary Tree Binary Tree Simplest tree structure
Every node has the property: children.
“Left child” and “Right child”

33. Representing Trees Linked Lists (ala LISP) (A (B X) (C X (D E))))
Arrays (for defined n-ary) trees
A B C B X C X X X X X D E
class treeNode<Type> { Type data;
LinkedList<TreeNode<Type>> children; }

34. Graph Algorithm Determine if a graph contains a “universal sink” -
A graph with at least one node with an in-degree of |V|-1, and an out-degree of 0.