1. RAIK 283: Data Structures & Algorithms
Decrease and Conquer I
Dr. Ying Lu
Design and Analysis of Algorithms – Chapter 4

2. RAIK 283: Data Structures & Algorithms
Giving credit where credit is due:
Most of the lecture notes are based on the slides from the Textbook’s companion website
Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University, Dr. Chuck Cusack, Hope College and İnanç Tahralı, Gebze Institute of Technology
I have modified many of their slides and added new slides.
Design and Analysis of Algorithms – Chapter 4

3. Design and Analysis of Algorithms – Chapter 4
Decrease and Conquer
Reduce a problem instance to a smaller instance of the same problem and extend solution
Solve the smaller instance
Extend solution of smaller instance to obtain solution to original problem
Also referred to as inductive, incremental approach or chip and conquer
Design and Analysis of Algorithms – Chapter 4

4. Examples of Decrease and Conquer
Decrease by one:
Insertion sort
Graph algorithm:
Topological sorting
Algorithms for generating permutations, subsets
Decrease by a constant factor
Binary search
Fake-coin problems
multiplication à la russe
Josephus problem
Variable-size decrease
Euclid’s algorithm
Selection by partition
Design and Analysis of Algorithms – Chapter 4

5. Insertion Sort A Decrease-by-One algorithm: Insertion-Sort(A[0…n-1]) {
Insert A[n-1] into the sorted list A[0…n-2]
return the sorted list A[0…n-1] }

6. Directed Acyclic Graphs
A directed acyclic graph or DAG is a directed graph with no directed cycles:
Design and Analysis of Algorithms – Chapter 4

7. Topological Sort of a DAG (I)
DAGs arise in many modeling problems, e.g.:
course prerequisite structure
food chains
A topological sort implies a partial ordering on the domain
Design and Analysis of Algorithms – Chapter 4

8. Topological Sort of a DAG (II)
Linear ordering of all vertices in graph G
such that for every edge (u, v)  G, the vertex, u, where the edge starts is listed before the vertex, v, where the edge ends.
cs101
math101
cs525
cs401
cs445
cs990
cs203
cs310
Design and Analysis of Algorithms – Chapter 4

9. Design and Analysis of Algorithms – Chapter 4
Topological Sort
cs101
cs203
math101
cs310
cs401
cs445
cs525
cs990
Design and Analysis of Algorithms – Chapter 4

10. Topological Sorting Algorithm (I)
1. Source removal algorithm
Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges
How would you find a source (or determine that such a vertex does not exist) in a digraph
represented by adjacency matrix?
represented by adjacency linked list?
Design and Analysis of Algorithms – Chapter 4

11. Topological Sorting Algorithm (I)
1. Source removal algorithm
Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges
How would you find a source (or determine that such a vertex does not exist) in a digraph
represented by adjacency matrix?
represented by adjacency linked list?
Θ(|V|2) using adjacency matrix
Θ(|V|+|E|) using adjacency linked lists
Design and Analysis of Algorithms – Chapter 4

12. Topological Sorting Algorithm (I)
1. Source removal (a Decrease-by-One) algorithm
Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges
cs101
cs203
math101
cs310
cs401
cs445
cs525
cs990
Design and Analysis of Algorithms – Chapter 4 12 12

13. Topological Sorting Algorithm (II)
2. DFS-based algorithm:
cs101
math101
cs525
cs401
cs445
cs990
cs203
cs310
Design and Analysis of Algorithms – Chapter 4

14. Topological Sorting Algorithm (II)
2. DFS-based algorithm:
DFS traversal noting order vertices are popped off stack
Reverse order solves topological sorting
Θ(V2) using adjacency matrix
Θ(V+E) using adjacency linked lists
Design and Analysis of Algorithms – Chapter 4

15. Design and Analysis of Algorithms – Chapter 4
In-Class Exercise
Page 143
Design and Analysis of Algorithms – Chapter 4

16. Design and Analysis of Algorithms – Chapter 4
Sum Rule
If a task can be done either in one of n1 ways or in one of n2 ways, where none of the set of n1 ways is the same as any of the set of n2 ways, then there are n1+n2 ways to do the task
In general,
If A1, A2, …, Am are disjoint finite sets, then the number of elements in the union of these sets is as follows
|A1⋃A2 ⋃… ⋃Am|=|A1|+|A2|+…+|Am|
Design and Analysis of Algorithms – Chapter 4

17. Design and Analysis of Algorithms – Chapter 4
Sum Rule
If A1, A2, …, Am are disjoint finite sets, then the number of elements in the union of these sets is as follows
|A1⋃A2 ⋃… ⋃Am|=|A1|+|A2|+…+|Am|
Design and Analysis of Algorithms – Chapter 4

18. Insertion Sort A Decrease-by-One algorithm: Insertion-Sort(A[0…n-1]) {
Insert A[n-1] into the sorted list A[0…n-2]
return the sorted list A[0…n-1] }

19. Topological Sorting Algorithm
Source removal (a Decrease-by-One) algorithm
Repeatedly identify and remove a source vertex, i.e., a vertex that has no incoming edges
cs101
cs203
math101
cs310
cs401
cs445
cs525
cs990
Design and Analysis of Algorithms – Chapter 4 19 19

20. Algorithms for Generating Combinatorial Objects
Definition :
Most important types of combinatorial objects
permutations
combinations
subsets of a given set
They typically arise in problems that require a consideration of different choices
TSP, Knapsack
To solve these problems need to generate combinatorial objects
Design and Analysis of Algorithms - Chapter 4 20 20

21. Generating Permutations
Assume the set whose elements need to be permuted is the set of integers from 1 to n
They can be interpreted as indices of elements in an n-element set {ai, …, an}
Design and Analysis of Algorithms - Chapter 4 21 21

22. Generating Permutations
Assume the set whose elements need to be permuted is the set of integers from 1 to n
They can be interpreted as indices of elements in an n-element set {ai, …, an}
What would the decrease-by-one technique suggest for the problem of generating all n! permutations?
Design and Analysis of Algorithms - Chapter 4 22 22

23. Generating Permutations
Approach :
The smaller-by-one problem is to generate all (n-1)! permutations
Assuming that the smaller problem is solved
We can get a solution to the larger one
by inserting n in each of the n possible positions among elements of every permutation of n-1 elements
Total number of all permutations will be n.(n-1) ! = n!
Design and Analysis of Algorithms - Chapter 4 23 23

24. Generating Permutations
We can insert n in the previously generated permutations
left to right
right to left
Design and Analysis of Algorithms - Chapter 4 24 24

25. Generating Permutations
minimal-change requirement
Each permutation differs from the previous one by exchanging only two elements
Benefical for algorithm’s speed
EX: Advantage of this order in TSP:
The length of the new tour can be calculated in constant time
By using the length of the previous tour
Tour Cost
a-b-c-d-e-f-g-h-a =22
a-b-c-d-e-f-h-g-a =25
Design and Analysis of Algorithms - Chapter 4 25 25

26. Generating Permutations
Approach :
Assume all permutations of {1,2,…,(n-1)} are available
Start with inserting n into 1,2,…,(n-1) by moving right to left
Then swicth direction every time a new permutation of {1,2,…,(n-1)} is processed
Design and Analysis of Algorithms - Chapter 2 26 26

27. Generating Permutations
This order satisfies minimal-change requirement
This approach requires that all the permutations of {1,2,…,(n-1)} are calculated already
Not easy to do!... (requires lots of space)
Design and Analysis of Algorithms - Chapter 4 27 27

28. Generating Permutations
Another way to get the same ordering :
Associate a direction with each component k in a permutation
Indicate such a direction by a small arrow
The component k is said to be mobile
if its arrow points to a smaller number adjacent to it
Design and Analysis of Algorithms - Chapter 4 28 28

29. Generating Permutations
Another way to get the same ordering :
Associate a direction with each component k in a permutation
Indicate such a direction by a small arrow
The component k is said to be mobile
if its arrow points to a smaller number adjacent to it
3 and 4 are mobile
2 and 1 are not
The following algorithm uses this notion
Design and Analysis of Algorithms - Chapter 4 29 29

30. Generating Permutations
ALGORITHM Johnson Trotter (n)
// Implements Johnson-Trotter algorithm for generating permutations
// Input : A positive integer n
// Output : A list of permutations of {1, … , n}
Initialize the first permitation with 1 2 … n
while there exists a mobile integer k do
find the largest mobile integer k
swap k and the adjacent integer its arrow points to
reverse the direction of all integers that are larger than k
Design and Analysis of Algorithms - Chapter 4 30 30

31. Generating Permutations
An application of Johnson Trotter algorithm :
Design and Analysis of Algorithms - Chapter 4 31 31

32. Design and Analysis of Algorithms – Chapter 4
In-class Exercise
Page 148 exercise 4.3.2
Generate all permutations of {1,2,3,4} by
a. the bottom-up minimal-change algorithm.
b. the Johnson-Trotter algorithm.
Design and Analysis of Algorithms – Chapter 4 32 32

33. Generating Permutations
The Johnson-Trotter algorithm does not produce permutations in lexicographical order
Example
Johnson-Trotter algorithm: 123, 132, 312, 321, 231, 213
Lexicographical order: 123, 132, 213, 231, 312, 321
Use an example: produce permutations {1,2,3,4,5} in lexicographical order, to get an idea for the algorithm
Design and Analysis of Algorithms – Chapter 4 33 33