1. Divide & Conquer  Themes  Reasoning about code (correctness and cost)  recursion, induction, and recurrence relations  Divide and Conquer  Examples  sorting (insertion sort & merge sort)  computing powers

2. Sorting  Given a list L = (L 1 … L n ) of elements from a totally ordered set (e.g. integers with ≤) return a list M = (M 1 … M n ) such that  (sorted M), i.e. M 1 ≤ M 2 ≤    ≤ M n  M = (permutation L), i.e. M i = L  (i) where  is a bijection from {1,…,n} to {1,…,n}  Use divide and conquer to derive several recursive algorithms and the principle of balance to obtain an efficient algorithm

3. sortedp  Predicate to test to see if a list is sorted  Should check to see that L is a list of integers  (x 1 x 2 … x n ) with x 1 ≤ x 2 ≤    ≤ x n  x i ≤ x j for 1 ≤ i < j ≤ n (defun sortedp (L) (cond ((equal L nil) t) ((equal (length L) 1) t) ((and (<= (first L) (second L)) (sortedp (rest L))))))

4. Correctness of sortedp  L = (x 1 x 2 … x n )  By induction on len(L)  Base cases L nil or of length 1  Show x i ≤ x j for 1 ≤ i < j ≤ n  x 1 ≤ x 2 and by induction x i ≤ x j for 2 ≤ i < j ≤ n  x 1 ≤ x 2 ≤ x j  x 1 ≤ x j for 2 ≤ j ≤ n

5. Precise Proof is Difficult  We take advantage of informal reasoning and names which provide intuitive meaning  L = (x 1 x 2 … x n )  Prove (fn123 L) = t if x i ≤ x j for 1 ≤ i < j ≤ n and nil otherwise using equational properties of fn13, fn14, fn15, fn17, fn81, fn100 and fn150 (defun fn123 (L) (fn13 ((fn81 L nil) t) ((fn81 (fn150 L) 1) t) ((fn82 (fn100 (fn14 L) (fn17 L)) (fn123 (fn15 L))))))

6. Insertion Sort  To sort x 1,…,x n,  recursively sort x 2,…,x n  insert x 1 into x 2,…,x n  (see code for details)  Prove correctness by induction

7. Insertion Sort (defun insertionsort (L) (if (equal L nil) nil (insert (first L) (insertionsort (rest L))))) (defun insert (x L) (cond ((equal L nil) (list x)) ((<= x (first L)) (cons x L)) ((cons (first L) (insert x (rest L))))))

8. Insertion Sort (Example)  (insertionsort ‘(7,6,5,4,3,2,1,0))  recursive call returns (0,1,2,3,4,5,6)  (insert 7 ‘(0,1,2,3,4,5,6))  (0,1,2,3,4,5,6,7)  Number of comparisons to sort inputs that are in reverse sorted order (worst case) C(n) = C(n-1) + (n-1) C(1) = 0

9. Correctness of insertionsort L = (L 1 … L n ) 1.(insertionsort L) terminates and returns a list of length n 2.(insertionsort L) is sorted 3.(insertionsort L) is a permutation of L Proof by induction on n 1.For n  0, recursive call with smaller n 2.Recursive call produces a sorted list of (rest L) by induction and (insert (first L) (insertionsort (rest L))) returns a sorted list (by induction) 3.Recursive call returns a permutation of (rest L) and insert returns a list containing (first L) and elements of (rest L)

10. Solve Recurrence for C(n)

11. Merge Sort  To sort x 1,…,x n,  recursively sort x 1,…,x a and x a+1,…,x n, where a = n/2  merge two sorted lists  Insertion sort is a special case where a=1

12. Mergesort (defun mergesort (L) (cond ((equal L nil) nil) ((equal (length L) 1) L) ((merge (mergesort (take (floor (length L) 2) L)) (mergesort (drop (floor (length L) 2) L))))))

13. Merge (defun merge (L M) (cond ((equal L nil) M) ((equal M nil) L) ((<= (first L) (first M)) (cons (first L) (merge (rest L) M))) ((cons (first M) (merge L (rest M))))))

14. Take and Drop (defun take (n L) (cond ((equal L nil) nil) ((equal n 0) nil) ((cons (first L) (take (- n 1) (rest L)))))) (defun drop (n L) (cond ((equal L nil) nil) ((equal n 0) L) ((drop (- n 1) (rest L)))))

15. Merge Sort (Example)  (7,6,5,4,3,2,1,0)  after recursive calls (4,5,6,7) and (0,1,2,3)  Number of comparisons needed to sort, worst case (the merged lists are perfectly interleaved) M(n) = 2M(n/2) + n-1 M(1) = 0  What is the best case (all in one list > other list)? M(n) = 2M(n/2) + n/2

16. Comparison of Insertion and Merge Sort  Count the number of comparisons for different n=2 k  M(n)/C(n)  0 as n increases  C(n) grows faster than M(n)  C(2n)/C(n)  4  So, apparently quadratic.  C(n) = Θ(n 2 )  M(2n)/M(n)  2 as n increases nM(n)C(n)M(n)/C(n) 2111 4460.667 812280.429 16321200.267 32804960.161 6419220160.095 128448181280.055

17. Solve Recurrence for M(n)

18. Mergesort What is wrong with the following? (defun mergesort (L) (if (equal L nil) nil (merge (mergesort (mytake (floor (length L) 2) L)) (mergesort (drop (floor (length L) 2) L)))))

19. Correctness of mergesort L = (L 1 … L n ) 1.(mergesort L) terminates and returns a list of length n 2.(mergesort L) is sorted 3.(mergesort L) is a permutation of L 4.Proof by induction on n 1.For n  2, recursive calls with smaller n 2.Recursive calls produce sorted lists by induction and merge produces a sorted list 3.Recursive calls contain lists whose concatenation is L and hence by induction return lists whose concatenation is L. merge returns a permutation of L

20. Properties of nth L = (L 1 … L len(L) ) 1.0 < n ≤ len(L), then (nth n L) = L n  Proof by induction on n 2.n > len(L), then (nth n L) = nil  Proof by induction on len(L) (defun nth (n L) (if (equal L nil) nil (if (equal n 1) (first L) (nth (- n 1) (rest L)))))

21. Properties of nth L = (L 1 … L len(L) ) 1.0 < n ≤ len(L), then (nth n L) = L n  Proof by induction on n 1.Base case. n = 1 return (first L) 2.By induction (nth (- n 1) (rest L)) returns the (n-1)st element of (rest L) which is the nth element of L (n > 1)

22. Properties of nth L = (L 1 … L len(L) ) 1.n > len(L), then (nth n L) = nil  Proof by induction on len(L) 1.Base case. Len(L) = 0. L = nil and return nil 2.n > len(L)  n-1 > len((rest L)), therefore by induction (nth (- n 1) (rest L)) returns nil

23. Properties of take and drop L = (L 1 … L len(L) ) 1.0 ≤ n ≤ len(L), then len((take n L)) = n  Proof by induction on n 2.n > len(L), then (take n L) = L  Proof by induction on len(L) 3.0 < k ≤ n ≤ len(L), then (nth k (take n L)) = L k  Proof by property of nth and induction on k 4.0 ≤ n ≤ len(L), then len((drop n L)) = len(L) – n 5.n > len(L), then (drop n L) = nil 6.0 ≤ n ≤ len(L), 0 < k ≤ len(L)-n, then (nth k (drop n L)) = L n+k 7.0 ≤ n ≤ len(L), (append (take n L) (drop n L)) = L

24. Properties of merge L = (L 1 … L m ) and M = (M 1 … M n ) Assume L is sorted and M is sorted (merge L M) is a list of length m+n (merge L M) is sorted (merge L M) is a permutation of (L 1 … L m M 1 … M n )

25. Exercise  Use check= to verify properties of take and drop for several examples  Use induction to prove the properties

26. Solution Property 1 L = (L 1 … L len(L) ) 1.0 ≤ n ≤ len(L), then len((take n L)) = n  Proof by induction on n 1.Base case. n = 0 return nil and len(nil) = 0. 2.Assume n > 0 and show that len((take n L)) = n. Assume that take returns a list of length k for k < n [Inductive hypothesis] 3.In this case take returns (cons (first L) (take (- n 1) (rest L))) and since n-1 ≤ len((rest L)) (take (- n 1) (rest L)) by induction is a list of length n-1. Therefore (cons (first L) (take (- n 1) (rest L))) has length (n-1) + 1 = n.

27. Solution Property 2 L = (L 1 … L len(L) ) 1.n > len(L), then (take n L) = L  Proof by induction on len(L) 1.Base case. len(L) = 0 and take returns L = nil independent of n. 2.Assume n > len(L) = and show that (take n L) = L. Assume that take returns M when len(M) < [Inductive hypothesis] 3.In this case take returns (cons (first L) (take (- n 1) (rest L)). By induction, (take (- n 1) (rest L)) returns (rest L) and therefore (take n L) returns (cons (first L) (rest L)) = L.

28. Solution Property 3 L = (L 1 … L len(L) ) 1.0 < k ≤ n ≤ len(L), then (nth k (take n L)) = L k  Proof by property of nth and induction on k 1.Base case. k = 1and by the first property of nth, (nth 1 (take n L)) = the first element of (take n L) which by the definition of take is the first element of L. 2.Assume the jth element of (take n L) = L j for j < k [Inductive hypothesis] and show that the kth element of (take n L) = L k. 3.By the first property of nth (nth k (take n L)) is the kth element of (take n L) and by the definition of take, that is the (k-1)st element of (take (- n 1) (rest L)) which by induction is the (k-1)st element of (rest L) which is the kth element of L.

29. Solution of Properties 4-7 L = (L 1 … L len(L) )  Properties 4-6 for drop are analogous to properties 1-3 for take and their proofs are similar to those for properties 1-3.  Property 7 follows from properties 3 and 6.

30. Computing Powers  Recursive definition:  a n = a  a n-1, n > 0  a 0 = 1  Number of multiplications  M(n) = M(n-1) + 1, M(0) = 0  M(n) = n

31. Exercise  Write a recursive ACL program to compute the power of a number using the recursive definition on the previous slide  Prove by induction on n that (power a n) = a n

32. Solution ; inputs a rational number a and a natural number n ; output a rational number equal to a^n (defun power (a n) (if (= n 0) 1 (* a (power a (- n 1))))) (check= (power 2 0) 1) (check= (power 2 10) 1024)

33. Solution Continued (defun power (a n) (if (= n 0) 1 (* a (power a (- n 1)))))  Base case n = 0  (power a 0) = 1 = a 0  Assume n > 0 and (power a (- n 1)) = a n-1 [Inductive Hypothesis]  (power a n) = (* a (power a (- n 1)) = a×a n-1 [by induction] = a n

34. Binary Powering (Recursive)  Binary powering  x 16 = (((x 2 ) 2 ) 2 ) 2, 16 = (10000) 2  x 23 = (((x 2 ) 2 x) 2 x) 2 x, 23 = (10111) 2  Recursive (right-left) x n = (x  n/2  ) 2  x (n % 2) M(n) = M(  n/2  ) + [n % 2] + 1

35. Exercise  Write a recursive ACL program to compute the power of a number using the binary powering algorithm  Prove by induction on n that (fastpower a n) = a n

36. Solution ; inputs a rational number a and a natural number n ; output a rational number equal to a^n (defun fastpower (a n) (if (= n 0) 1 (let ((b (fastpower a (floor n 2)))) (if (evenp n) (* b b) (* b b a))))) (check= (power 2 0) 1) (check= (fastpower 2 16) 65536) (check= (fastpower 2 15) 32768)

37. Solution Continued  Base case n = 0  (fastpower a 0) = 1 = a 0  Assume n > 0 and (fastpower a k) = a k for k < n [Inductive Hypothesis]  Even case n = 2k  (fastpower a n) = (* (fastpower a k) (fastpower a k)) = a k ×a k [by induction] = a 2k = a n  Odd case n = 2k+1  (fastpower a n) = (* (fastpower a k) (fastpower a k) a) = a k ×a k ×a [by induction] = a 2k+1 = a n

38. Binary Powering  Number of multiplications  Let (n) = number of 1bits in binary representation of n  num bits =  lg(n)  +1  M(n) =  lg(n)  + (n)  (n) ≤ lg(n) =>  M(n) ≤ 2lg(n)

39. Asymptotic Growth Rates f(n) = O(g(n)) [grows at the same rate or slower] There exists positive constants c and n 0 such that f(n)  c g(n) for all n  n 0 Ignore constants and low order terms

40. Asymptotic Growth Rates f(n) = o(g(n)) [grows slower] f(n) = O(g(n)) and g(n)  O(f(n)) lim n  f(n)/g(n) = 0 f(n) =  (g(n)) [grows at the same rate] f(n) = O(g(n)) and g(n) = O(f(n))

41. Asymptotic Growth Rates  (log(n)) – logarithmic [log(2n)/log(n) = 1 + log(2)/log(n)]  (n) – linear [double input  double output]  (n 2 ) – quadratic [double input  quadruple output]  (n 3 ) – cubit [double input  output increases by factor of 8]  (n k ) – polynomial of degree k  (c n ) – exponential [double input  square output]

42. Merge Sort and Insertion Sort Insertion Sort T I (n) = T I (n-1) +  (n) =  (n 2 ) [worst case] T I (n) = T I (n-1) +  (1) =  (n) [best case] Merge Sort T M (n) = 2T M (n/2) +  (n) =  (nlogn) [worst case] T M (n) = 2T M (n/2) +  (n) =  (nlogn) [best case]

43. Divide & Conquer Recurrence Assume T(n) = aT(n/b) +  (n) T(n) =  (n) [a < b] T(n) =  (nlog(n)) [a = b] T(n) =  (n log b (a) ) [a > b]

44. Karatsuba’s Algorithm Using the classical pen and paper algorithm two n digit integers can be multiplied in O(n 2 ) operations. Karatsuba came up with a faster algorithm. Let A and B be two integers with A = A 1 10 k + A 0, A 0 < 10 k B = B 1 10 k + B 0, B 0 < 10 k C = A*B = (A 1 10 k + A 0 )(B 1 10 k + B 0 ) = A 1 B 1 10 2k + (A 1 B 0 + A 0 B 1 )10 k + A 0 B 0 Instead this can be computed with 3 multiplications T 0 = A 0 B 0 T 1 = (A 1 + A 0 )(B 1 + B 0 ) T 2 = A 1 B 1 C = T 2 10 2k + (T 1 - T 0 - T 2 )10 k + T 0

45. Complexity of Karatsuba’s Algorithm Let T(n) be the time to compute the product of two n-digit numbers using Karatsuba’s algorithm. Assume n = 2 k. T(n) =  (n lg(3) ), lg(3)  1.58 T(n)  3T(n/2) + cn  3(3T(n/4) + c(n/2)) + cn = 3 2 T(n/2 2 ) + cn(3/2 + 1)  3 2 (3T(n/2 3 ) + c(n/4)) + cn(3/2 + 1) = 3 3 T(n/2 3 ) + cn(3 2 /2 2 + 3/2 + 1) …  3 i T(n/2 i ) + cn(3 i-1 /2 i-1 + … + 3/2 + 1)...  cn[((3/2) k - 1)/(3/2 -1)] --- Assuming T(1)  c  2c(3 k - 2 k )  2c3 lg(n) = 2cn lg(3)

46. Identities for Sums

47. Some Common Sums

48. Arithmetic Series  Sum of consecutive integers (written slightly differently):

49. Arithmetic Series (Proof)

50. Geometric Series = 1 + x + x 2 + … + x n-1 =

51. Geometric Series (Proof)

52. Floor and Ceiling  Let x be a real number  The floor of x,  x , is the largest integer less than or equal to x  If an integer k satisfies k  x < k+1, k =  x   E.G.  3.7  = 3,  3  = 3  The ceiling of x,  x , is the smallest integer greater than or equal to x  If an integer k satisfies k-1 < x  k, k =  x   E.G.  3.7  = 4,  3  = 3

53. logarithm  y = log b (x)  b y = x  Two important cases  ln(x) = log e (x)  lg(x) = log 2 (x) [frequently occurs in CS]  Properties  log(cd) = log(c) + log(d)  log(c x ) = xlog(c)  log b (b x ) = x = b logb(x)  d ln(x)/dx = 1/x

54. logarithm  2 k  x < 2 k+1  k =  lg(x)   E.G. 16  25 < 32  4  lg(25) < 5  lg(25)  4.64  Change of base  log c (x) = log b (x) / log b (c) Proof. y = log c (x)  c y = x  ylog b (c) = log b (x)  y = log b (x) / log b (c)