1. Lecture 4: Divide-and-Conquer

2. Steps in Divide and Conquer As its name implies divide-and-conquer involves dividing a problem into smaller problems that can be more easily solved. While the specifics vary from one application to another, divide-and-conquer always includes the following three steps in some form: Divide - Typically this steps involves splitting one problem into two problems of approximately 1/2 the size of the original problem. Conquer - The divide step is repeated (usually recursively) until individual problem sizes are small enough to be solved (conquered) directly. Recombine - The solution to the original problem is obtained by combining all the solutions to the sub-problems. Divide and Conquer is not applicable to every problem class. Even when D&C works it may not provide for an efficient solution.

3. Binary Search - Iterative Version procedure bin1(n:integer,x:keytype,S:list,loc:index)is lo,hi,mid : index; begin lo:=1; hi:=n; loc:=0; while lo<=hi and loc=0 loop mid:=(lo+hi)/2; if x=S(mid) then loc:=mid; elsif x<S(mid) then hi:=mid-1; else lo:=mid+1; end if; end loop; end bin1; S 0 10 1 12 2 15 3 19 4 19 5 24 6 39 7 45 8 53 9 77 x=42 lo=1 hi=9 mid=4 S(4)=19 lo=5 hi=9 mid=7 S(7)=45 lo=5 hi=6 mid=5 S(5)=24 lo=6 hi=6 mid=6 S(6)=39 lo=7 hi=6 T(n) = C + lg 2 n D -> O(lg 2 n)

4. Binary Search - Recursive Version function location(lo,hi : index) return index is begin if lo>hi then return 0; else mid:=(lo+hi)/2; if x=S(mid) then return mid; elsif x<S(mid) then return location(lo,mid-1); else return location(mid+1,hi); end if; end location; S 0 10 1 12 2 15 3 19 4 19 5 24 6 39 7 45 8 53 9 77 x=42 lo=0 hi=9 mid=4 S(4)=19 lo=5 hi=9 mid=7 S(7)=45 lo=5 hi=6 mid=5 S(5)=24 lo=6 hi=6 mid=6 S(6)=39 lo=7 hi=6 return 0

5. Analysis of Function location( ) C n<=1 T(n) = T(n/2) + D otherwise T(n/2)=T(n/4) + D T(n)=(T(n/4) + D) + D T(n)=T(n/4) + 2D T(n)=T(n/8) + 3D : T(n)=T(n/2 k ) + kD T(n)=T(1) + (lg 2 n)D T(n)=C + (lg 2 n)D T(n) -> O(lg 2 n) We need a closed form expression for T(n) that does not contain a term involving T. We assume that n>1 and note that we need to replace T(n/2) with an explicit function of n. We can use the recurrence relation to express the term T(n/2) in another form that can, in turn be substituted back into our expression for T(n). Eventually we can write a general expression for the kth substitution. Then we can determine a value for k (in terms of n) that sets the parameter of T equal to one. This lets us substitute T(1)=C in our recurrence and replace any other occurrences of k with terms involving n. We can then determine the order of complexity.

6. Mergesort procedure msort(n:integer; S:list) is h: constant :=n/2; m: constant :=n-h; U,V : list; begin if n>1 then U(1..h):=S(1..h); V(1..m):=S(h+1..n); msort(h,U); msort(m,V); merge(h,m,U,V,S); end if; end msort; procedure merge(h,m:integer; U,V,S:list); i,j,k : integer; begin i:=1; j:=1; k:=1; while i<=h and j<=m loop if U(i)<V(j) then S(k):=U(i); i:=i+1; else S(k):=V(j); j:=j+1; end if; k:=k+1; end loop; if i>h then S(k..h+m):=V(j..m); else S(k..h+m):=U(i..h); end if; end merge; Write the recurrence relation for the complexity of msort and then solve it.

7. Quicksort procedure quicksort(lo,hi: integer) is pivot : integer; begin if hi>lo then partition(lo,hi,pivot); quicksort(lo,pivot-1); quicksort(pivot,+1,hi); end if; end quicksort; procedure partition(lo,hi,pivot)is j,item : integer; begin item := S(lo); j:= lo; for i in lo+1..hi loop if S(i)<item then j:=j+1; swap(S(i),S(j)); end if; end loop; pivot:=j swap(S(lo),S(pivot)); end partition; Where is the recombine step in the quicksort divide-and-conquer algorithm?

8. Quicksort Example ij1234567891011121314 2169857463547605 4265897463547605 6365497863547605 8465437869547605 9565435869747605 10665435469787605 13765435409787665 14865435405787669 855435406787666 pivot value items being swapped new sublists for next pass of quicksort

9. Quicksort: Worst-Case Performance What does it mean? An analysis shows that the worst-case performance for quicksort( ) is order O(n 2 ). However, quicksort performs much better than this in practice. Does this mean that algorithm analysis is a waste of effort? Not really. In addition to showing worst-case performance, algorithm analysis provides us with a better understanding of the conditions under which an algorithm performs well. We can see that the quicksort algorithm performs worst on a sorted list and performs best on a randomly distributed list (at least one in which the pivot value is always near the median value of the sublist being partitioned). Algorithm analysis helps us determine which algorithm is best for a particular application, as well as suggesting ways to modify an algorithm to improve its performance. Can you think of a simple way to improve quicksort( )?

10. Searching and sorting problems are obvious candidates for D&C but the Divide-and-Conquer problem-solving method gets used in other ways you might not expect. Other Applications for D&C Game Strategy Computing numeric roots Fast Exponentiation

11. Suppose that we need to compute the value of a n for some reasonably large n. Such problems occur in primality testing for cryptography. The simplest algorithm performs n − 1 multiplications, by computing a × a ×... × a. However, we can do better by observing that n = ⌊ n/2 ⌋ + ⌈ n/2 ⌉. If n is even, then a n = (a n/2 )2. If n is odd, then a n = a(a ⌊ n/2 ⌋ )2. In either case, we have halved the size of our exponent at the cost of at most two multiplications, so O(lg 2 n) multiplications suffice to compute the final value. function power(a, n) if (n = 0) return(1) x = power(a, ⌊ n/2 ⌋ ) if (n is even) return(x 2 ) else return(a * x 2 ) Modular Exponentiation Introduction to Algorithms - Steven Skiena

12. The square root of n is the number r such that r 2 = n. Square root computations are performed inside every pocket calculator – but how? Observe that the square root of n ≥ 1 must be at least 1 and at most n. Let l = 1 and r = n. Consider the midpoint of this interval, m = (l + r)/2. How does m 2 compare to n? If n ≥ m 2, then the square root must be greater than m, so the algorithm repeats with l = m. If n < m 2, then the square root must be less than m, so the algorithm repeats with r = m. Either way, we have halved the interval with only one comparison. Therefore, after only lg 2 n rounds we will have identified the square root to within ±1. Square Root (and other roots) of a Number

13. Twenty Questions Player One - Picks a word. Player Two - Asks a series of YES or NO questions, attempting to guess the word in <20 tries. alphabetical version of Twenty Questions

14. Computing the Median Computing the median of n numbers is easy: just sort them. The drawback is that this takes O(n log 2 n) time, whereas we would ideally like something linear. We have reason to be hopeful, because sorting is doing far more work than we really need. We just want the middle element and don't care about the relative ordering of the rest of them. When looking for a recursive solution, it is paradoxically often easier to work with a more general version of the problem for the simple reason that this gives a more powerful step to recurse upon. In our case, the generalization we will consider is selection. SELECTION Input: A list of numbers S; an integer k Output: The kth smallest element of S For instance, if k = 1, the minimum of S is sought, whereas if k = [ |S|/2 ], it is the median. Divide-and-Conquer Algorithms - Vazirani

15. Chapter 2 Divide-and-conquer algorithms – see readings http://www.cs.berkeley.edu/~vazirani/ The K-Median Algorithm (Selection)

16. 6784023458148973 Finding the Median 6784023458148973 SLSL SMSM SRSR 67844584897 3 67844584897 5 SLSL SMSM SRSR 4444

17. Summary Divide-and-Conquer (actually three steps) 1. divide 2. conquer 3. recombine Most Naturally Implemented using Recursion (although iteration possible) Analyze performance by solving Recurrence Relations Worst-Case Performance may not be typical (or even possible) performance. Divide-and-Conquer is embedded in Arithmetic Processors/Calculators Cryptography relies on D&C Methods Game Strategy can be based on D&C Using D&C Selection Algorithm to Find the Median