Theory of Algorithms: Decrease and Conquer James Gain and Edwin Blake {jgain | Department of Computer Science University of Cape Town August - October 2004

Objectives lTo introduce the decrease-and-conquer mind set lTo show a variety of decrease-and-conquer solutions:  Depth-First Graph Traversal  Breadth-First Graph Traversal  Fake-Coin Problem  Interpolation Search lTo discuss the strengths and weaknesses of a decrease-and-conquer strategy

Decrease and Conquer 1.Reduce problem instance to smaller instance of the same problem and extend solution 2.Solve smaller instance 3.Extend solution of smaller instance to obtain solution to original problem lAlso called inductive or incremental lUnlike Divide-and-Conquer don’t work equally on both subproblems (e.g. Binary Search) SUBPROBLEM OF SIZE n-1 A SOLUTION TO THE ORIGINAL PROBLEM A SOLUTION TO SUBPROBLEM A PROBLEM OF SIZE n

Flavours of Decrease and Conquer lDecrease by a constant (usually 1): instance is reduced by the same constant on each iteration  Insertion sort  Graph Searching: DFS, BFS  Topological sorting  Generating combinatorials lDecrease by a constant factor (usually 2): instance is reduced by same multiple on each iteration  Binary search  Fake-coin problem lVariable-size decrease: size reduction pattern varies from one iteration to the next  Euclid’s algorithm  Interpolation Search

Exercise: Spot the Difference lProblem: Derive algorithms for computing a n using: 1.Brute Force 2.Divide and conquer 3.Decrease by one 4.Decrease by constant factor (halve the problem size) lHint: each can be described in a single line

Graph Traversal lMany problems require processing all graph vertices in a systematic fashion lData Structures Reminder: lGraph traversal strategies:  Depth-first search (traversal for the Brave)  Breadth-first search (traversal for the Cautious) ab cd abcd a0111 b1001 c1001 d1110 a b c d a bc d d ad ab c

Depth-First Search lExplore graph always moving away from last visited vertex lSimilar to preorder tree traversal l DFS(G): G = (V, E) count  0 mark each vertex as 0 FOR each vertex v  V DO IF v is marked as 0 dfs(v) dfs(v): count  count + 1 mark v with count FOR each vertex w adjacent to v DO IF w is marked as 0 dfs(w)

Example: DFS ab ef cd gh Traversal Stack: (pre = push, post = pop) 1 a 8 2 b 7 3 f 2 4 e 1 5 g 6 6 c 5 7 d 4 8 h 3 Push order: a b f e g c d h Pop order:e f h d c g b a

DFS Forest lDFS Forest: A graph representing the traversal structure lTypes of Edges:  Tree edges: edge to next vertex in traversal  Back edges: edge in graph to ancestor nodes  Forward edges: edge in graph to descendants (digraphs only)  Cross edges: none of the above ab ef cd gh a bef cdgh Backedge

Notes on Depth-First Search lImplementable with different graph structures:  Adjacency matrices:  (V 2 )  Adjacency linked lists:  (V+E) lYields two orderings:  preorder: as vertices are 1 st encountered (pushed)  postorder: as vertices become dead-ends (popped) lApplications:  Checking connectivity, finding connected components  Checking acyclicity  Searching state-space of problems for solution (AI)

Breadth-First Search lMove across to all neighbours of the last visited vertex lSimilar to level-by-level tree traversals lInstead of a stack, breadth- first uses a queue BFS(G): G = (V, E) count  0 mark each vertex as 0 FOR each vertex v  V DO bfs(v) bfs(v): count  count + 1 mark v with count initialize queue with v WHILE queue not empty DO a  front of queue FOR each vertex w adjacent to a DO IF w is marked as 0 count  count + 1 mark w with count add w to queue remove a from queue

Example: BFS ab ef cd gh Traversal Queue: a 1 b 2 e 3 f 4 g 5 c 6 h 7 d 8 Order: a b e f g c h d a b e f cd g h Crossedge

Notes on Breadth First Search lBFS has same efficiency as DFS and can be implemented with:  Adjacency matrices:  (V 2 )  Adjacency linked lists:  (V+E) lYields single ordering of vertices lApplications: same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges

The Fake-Coin Problem: Decrease by a Constant Factor lProblem:  Among n identical looking coins, one is a fake (and weighs less)  We have a balance scale which can compare any two sets of coins lAlgorithm:  Divide into two size  n/2  piles (keeping a coin aside if n is odd)  If they weigh the same then the extra coin is fake  Otherwise proceed recursively with the lighter pile lEfficiency:  W(n) = W(  n/2  ) + 1 for n > 1  W(n) =  log 2 n  =  (log 2 n) lBut there is a better  (log 3 n) algorithm

Euclid’s GCD: Variable-Size Decrease lProblem:  Greatest Common Divisor of two integers m and n is the largest integer that divides both exactly lAlternative Solutions:  Consecutive integer checking (brute force)  Identify common prime factors (transform and conquer) lEuclid’s Solution:  gcd(m, n) = gcd(n, m mod n)  gcd(m, 0) = m  Right-side args are smaller by neither a constant size nor factor lExample:  gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

Interpolation Search: Variable-Size Decrease lMimics the way humans search through a phone book (look near the beginning for ‘Brown’) lAssumes that values between the leftmost (A[l]) and rightmost (A[r]) list elements increase linearly lAlgorithm (key = v, find search index = i):  Binary search with floating variable at index i  Setup straight line through (l, A[l]) and (r, A[r])  Find point P = (x, y) on line at y = v, then i = x  x = l + (v - A[l])(r - l) / (A[r] - A[l]) lEfficiency:  Average =  (log log n + 1), Worst =  (n) lir A[l] v A[r] index value

Strengths and Weaknesses of Decrease-and-Conquer Strengths:  Can be implemented either top down (recursively) or bottom up (without recursion)  Often very efficienct (possibly  (log n) )  Leads to a powerful form of graph traversal (Breadth and Depth First Search) ûWeaknesses:  Less widely applicable (especially decrease by a constant factor)