1. Decrease and Conquer Decrease and conquer technique Insertion sort
DFS and BFS
Topological sort

2. Expected Outcomes Students should be able to
Explain the idea and steps of decrease and conquer
Explain the ideas of insertion sort, DFS, BFS, topological sort
Analyze the time complexity of the above algorithms

3. Decrease and Conquer
Exploring the relationship between a solution to a given instance of (e.g., P(n) )a problem and a solution to a smaller instance (e.g., P(n/2) or P(n) )of the same problem.
Use top down(recursive) or bottom up (iterative) to solve the problem.
Example, n!
A top down (recursive) solution
A bottom up (iterative) solution

4. Examples of Decrease and Conquer
Decrease by a constant: the size of the problem is reduced by the same constant on each iteration/recursion of the algorithm.
Insertion sort
Graph search algorithms:
DFS
BFS
Algorithms for generating permutations, subsets
Decrease by a constant factor: the size of the problem is reduced by the same constant factor on each iteration/recursion of the algorithm.
Binary search
Fake-coin problems
Variable-size decrease: the size reduction pattern varies from one iteration of an algorithm to another.
Eucd’s algorithm

5. A Typical Decrease by One Technique
subproblem
of size n-1
a solution to the
a solution to
the original problem
a problem of size n
e.g., n!

6. A Typical Decrease by a Constant Factor (half) Technique
subproblem
of size n/2
a solution to the
a solution to
the original problem
a problem of size n
e.g., Binary search

7. A Typical Divide and Conquer Technique
subproblem 2
of size n/2
subproblem 1
a solution to
the original problem
a problem of size n
e.g., mergesort

8. What’s the Difference?
Consider the problem of exponentiation: Compute an
Decrease by one
Bottom-up: iterative (brute Force)
Top-down:recursive
Divide and conquer:
Decrease by a constant factor:
an= a*a*a*a*...*a
an= an* a if n > 1
= a if n = 1
an= a  n/2  * a  n/2  if n > 1
= a if n = 1
an = (an/2 ) if n is even and positive
= (a(n)/2 ) 2 * a if n is odd and > 1
= a if n = 1

9. Insertion Sort A decrease by one algorithm
A bottom-up (iterative) solution
A top-down (recursive) solution

10. Insertion Sort: an Iterative Solution
ALGORITHM InsertionSortIter(A[0..n])
//An iterative implementation of insertion sort
//Input: An array A[0..n] of n orderable elements
//Output: Array A[0..n] sorted in nondecreasing order
for i  1 to n – 1 do // i: the index of the first element of the unsorted part.
key = A[i]
for j  i – 1 to 0 do // j: the index of the sorted part of the array
if (key< A[j]) //compare the key with each element in the sorted part
A[j+1]  A[j]
A[j +1]  key //insert the key to the sorted part of the array
Efficiency?

11. Insertion Sort: a Recursive Solution
ALGORITHM InsertionSortRecur(A[0..n], n)
//A recursive implementation of insertion sort
//Input: An array A[0..n] of n orderable elements
//Output: Array A[0..n] sorted in nondecreasing order
if n > 1
InsertionSortRecur(A[0..n-2], n-2)
Insert(A[0..n-2], n) //insert A[n] to A[0..n-2]
ALGORITHM Insert(A[0..m], m)
//Insert A[m] to the sorted subarray A[0..m] of A[0..n]
//Input: A subarray A[0..m] of A[0..n]
//Output: Array A[0..m] sorted in nondecreasing order
key = A[m]
for j  m – 1 to 0 do
if (key< A[j])
A[j+1]  A[j]
else
break
A[j +1]  key
Index of the last element
Index of the element/key to be inserted.
Recurrence relation?

12. Exercises
Design a recursive decrease-by-one algorithm for finding the position of the largest element in an array of n real numbers.
Determine the time efficiency of this algorithm and compare it with that of the brute-force algorithm for the same problem.

13. Graph Traversal Graph traversal algorithms:
Many problems require processing all graph vertices in a systematic fashion
Graph traversal algorithms:
Depth-first search
Breadth-first search

14. Depth-first search The idea Similar to preorder tree traversals
traverse “deeper” whenever possible.
On each iteration, the algorithm proceeds to an unvisited vertex that is adjacent to the one it is currently in. If there are more than one neighbors, break the tie by the alphabetic order of the vertices.
When reaching a dead end ( a vertex with no adjacent unvisited vertices), the algorithm backs up one edge to the parent and tries to continue visiting unvisited vertices from there.
The algorithm halts after backing up to the starting vertex, with the latter being a dead end.
Similar to preorder tree traversals
It’s convenient to use a stack to trace the operation of depth-first search.
Push a vertex onto the stack when the vertex is reached for the first time.
Pop a vertex off the stack when it becomes a dead end.

15. Example – undirected graphsb e f c d g h
Traversal using alphabetical order of vertices. Work through this example in detail,
showing the traversal by highghting edges on the graph and showing how the stack
evolves:
8 h 3
7 d stack shown growing upwards
4 e c number on left is order that vertex was pushed onto stack
3 f g number on right is order that vertex was popped from stack
2 b overlap is because after e, f are popped off stack, g and c
1 a are pushed onto stack in their former locations.
order pushed onto stack: a b f e g c d h
order popped from stack: e f h d c g b a
* show dfs tree as it gets constructed, back edges
Depth-first traversal:Give the order in which the vertices were reached.
a b f e g c d h

16. Depth-first search (DFS)
Pseudocode for Depth-first-search of graph G=(V,E)
DFS(G)
// Implements a DFS traversal of a given Graph
// Input: Graph G = {V, E}
// Output: Graph G with its vertices marked with consecutive integers in the order they’ve been first encounter by the DFS traversal mark each vertex in V with 0 as a mark of being “unvisited”
count  0 //visiting sequence number
for each vertex v∈ V do
if v is marked with 0 //w has not been visited yet.
dfs(v)
dfs(v)
//Use depth-first to visit a connected component starting from vertex v.
count  count + 1
mark v with count //visit vertex v
for each vertex w in V adjacent to v do
if w is marked with 0 //w has not been visited yet.
dfs(w)

17. Another Algorithm: Four Arrays for DFS Algorithm
color[u]: the color of each vertex visited (The state of a vertex, u, is stored in a color variable as follows:)
Color[u] = WHITE means undiscovered
Color[u] = GRAY means discovered but not finished processing
Color[u] = BLACK means finished processing.
[u]: the predecessor of u, indicating the vertex from which u is discovered. (depth-first search uses π[v] to record the parent of vertex v. We have π[v] = NIL if and only if vertex v is the root of a depth-first tree.)
d[u]: discovery time, a counter indicating when vertex u is discovered. (When vertex v is changed from white to gray the time is recorded in d[v]. )
f[u]: finishing time, a counter indicating when the processing of vertex u (and the processing of all its descendants ) is finished. (When vertex v is changed from gray to black the time is recorded in f[v].)

18. DFS Algorithm
DFS (V, E) 1. for each vertex u in V[G] 2. do color[u] ← WHITE 3. π[u] ← NIL 4. time ← 0 5. for each vertex u in V[G] 6. do if color[u] ← WHITE 7. then DFS-Visit(u) ▷ build a new DFS-tree from u

19. DFS Algorithm (Continued)
DFS-Visit(u) 1. color[u] ← GRAY ▷ discover u 2. time ← time d[u] ← time 4. for each vertex v adjacent to u ▷ explore (u, v) 5. do if color[v] ← WHITE 6. then π[v] ← u 7. DFS-Visit(v) 8. color[u] ← BLACK 9. time ← time f[u] ← time ▷ we are done with u

20. DFS Algorithm (Example)1/ a d b c g e f

21. Example(continued) 1/ a d 2/ b c g e f

22. Example(continued) 1/ a d 2/ b 3/ c g e f

23. Example(Continued) 1/ a d 2/ b 3/ c g e f 4/

24. Example(Continued) 1/ a d 2/ b 3/ c g e f
4/5

25. Example(Continued) 1/ a d 2/ b
3/6 c g e f
4/5

26. Example(Continued) 1/ a d
2/7 b
3/6 c g e f
4/5

27. Example(Continued)
1/8 a d
2/7 b
3/6 c g e f
4/5

28. Example(Continued)
1/8 a 9/ d
2/7 b
3/6 c g e f
4/5

29. Example(Continued)
1/8 a 9/ d
2/7 b
3/6 c g e
10/ f
4/5

30. Example(Continued)
1/8 a 9/ d
2/7 b
3/6 c
10/11 g e f
4/5

31. Example(continued)
1/8 a 9/ d
2/7 b
3/6
12/13 c g e
10/11 f
4/5

32. Example(Continued) a b f e c g d
2/7
1/8
3/6
4/5
9/14
10/11
12/13 

33. What does DFS do? Given a graph G, it traverses all vertices of G and
Constructed a collection of rooted trees together with a set of source vertices (roots)
Outputs two arrays d[ ]/f[ ]
Note: forest is stored in array [ ], the [ ] value of a root is null.
DFS forest: DFS creates a forest F = (V, Ef), where
Ef = {([u], u)| where [ ] is computed in the DFS-VISIT calls}

34. Properties of DFS
Definition: in a rooted forest, on the simple path from the root to each vertex, a vertex u is called ancestor of all the vertices following it, and descendant of all the vertices preceding it.
Note: u = [v] if and only if DFS-VISIT(v) was called during a search of u’s adjacency st. u is called the parent of v. So:
vertex v is a descendant of vertex u in the depth-first forest if and only if v is discovered during the time in which u is gray.

35. Example a b f e c g d
2/7
1/8
3/6
4/5
9/14
10/11
12/13 

36. Properties of DFS Parenthesis Theorem:
In any DFS of a graph, for each pair of vertices u, v, exactly one of the following conditions holds:
u is a descendant of v, and [d[u], f[u]] is a subinterval of [d[v], f[v]].
u is an ancestor of v, and [d[v], f[v]] is a subinterval of [d[u], f[u]].
Neither u is a descendant of v nor v is a descendant of u, and [d[u], f[u]] and [d[v], f[v]] are disjoint.
The nth Catalan Number:

37. Nesting of descendants’ intervals
Corollary: vertex v is a proper descendant of vertex u in the depth-first forest for a graph G if and only if d[u]< d[v] < f[v] < f[u].

38. White-path theorem Theorem:
In a depth-first forest of a graph G = (V, E), vertex v is a descendant of u if and only if at time d[u], vertex v can be reached from u along a path consisting entirely of white vertices.

39. Example – directed graphb c d e f g h
Depth-first traversal: Give the order in which the vertices were reached.
a b f g h e c d

40. Depth-first search: Notes
DFS can be implemented with graphs represented as:
Adjacency matrices: Θ(|V |2)
Adjacency linked lists: Θ(|V |+|E| )

41. Applications of DFS
Find the strongly connected components of a directed graph.
Find the articulation points of an undirected graph
Topological sort of a directed graph
Classification of edges.
Verify if an undirected graph is connected or not.
Verify if a directed graph is semi-connected or not.
Verify if a directed graph is singly connected or not. ……

42. Breadth-first search (BFS)
Archetype for Prim’s minimum-spanning-tree algorithm
Archetype for Dijkstra’s single-source shortest-paths algorithm
The idea
Traverse “wider” whenever possible.
Discover all vertices at distance k from s (on level k) before discovering any vertices at distance k +1 (at level k+1)
Similar to level-by-level tree traversals
Instead of a stack, breadth-first uses a queue.

43. Example – undirected graphb e f c d g h
Breadth-first traversal:
a b e f g c h d

44. Breadth-first search algorithm
BFS(G)
count  0
mark each vertex with 0
for each vertex v∈ V do
if v is marked with 0
bfs(v)
bfs(v)
count  count + 1
mark v with count //visit v
initiaze queue with v //enqueue
while queue is not empty do
a  front of queue //dequeue
for each vertex w adjacent to a do
if w is marked with 0 //w hasn’t been visited.
mark w with count //visit w
add w to the end of the queue //enqueue
remove a from the front of the queue

45. Example – directed graphb c d e f g h
Breadth-first traversal:
a b e f g h c d

46. Another Implementation: The Breadth-First Search
The idea of the BFS:
Visit the vertices as follows:
Visit all vertices directly connected to starting vertex
Visit all vertices that are two edges away from the starting vertex
Visit all vertices that are three edges away from the starting vertex
… …
Initially, s is made GRAY, others are colored WHITE.
When a gray vertex is processed, its color is changed to black, and the color of all white neighbors is changed to gray.
Gray vertices are kept in a queue Q.

47. What does the BFS do? Given a graph G = (V, E), the BFS returns:
The shortest distance d[v] from s to v
The predecessor or parent [v], which is used to derive a shortest path from s to vertex v.
In addition to the two arrays d[v] and [v], BFS also uses another array color[v], which has three possible values:
WHITE: represented “undiscovered” vertices;
GRAY: represented “discovered” but not “processed” vertices;
BLACK: represented “processed” vertices.

48. The Breadth-First Search (more details)
G is given by its adjacency-sts.
Initiazation:
First Part: nes 1 – 4
Second Part: nes 5 - 9
Main Part: nes 10 – 18
Enqueue(Q, v): add a vertex v to the end of the queue Q
Dequeue(Q): Extract the first vertex in the queue Q

49. Breadth-first search: Notes
BFS has same efficiency as DFS and can be implemented with graphs represented as:
Adjacency matrices: Θ(|V |2)
Adjacency linked lists: Θ(|V |+|E| )
Applications:
checking connectivity
finding connected components
finding paths between two vertices with the smallest number of edges
Use a tree.

50. Is the problem solvable if the digraph contains a cycle?
Topological Sorting
Problem: Given a Directed Acycc Graph(DAG) G = (V, E), find a near ordering of all its vertices such that if G contains an edge (u, v), then u appears before v in the ordering.
Example: Give an order of the courses so that the prerequisites are met. C4 C1 C3
Is the problem solvable if the digraph contains a cycle? C5 C2

51. The DFS-Based Algorithm
DFS traversal noting the order in which vertices are popped off stack (the order in which the dead end vertices appear)
Reverse the above order.
Questions
Can we use the order in which vertices are pushed onto the DFS stack (instead of the order they are popped off it) to solve the topological sorting problem?
Does it matter from which node we start?
Θ(|V |+|E| ) using adjacency nked sts

52. The Source Removal Algorithms
Based on a direct implementation of the decrease-by-one techniques.
Repeatedly identify and remove a source vertex, ie, a vertex that has no incoming edges, and delete it along with all the edges outgoing from it.