CSC 413/513: Intro to Algorithms Merge Sort Solving Recurrences

Updates l Homework 1 posted n Due on Sep-01 l Office hours have been updated n MW: 1:00-2:15 n Tues: 1:00-2:30 n Thurs: 2:00-3:00 l Note taker?

Review: Asymptotic Notation l Upper Bound Notation: n f(n) is O(g(n)) if there exist positive constants c and n 0 such that f(n)  c  g(n) for all n  n 0 n Formally, O(g(n)) = { f(n):  positive constants c and n 0 such that f(n)  c  g(n)  n  n 0 l Big O fact: n A polynomial of degree k is O(n k )

Review: Asymptotic Notation l Asymptotic lower bound: n f(n) is  (g(n)) if  positive constants c and n 0 such that 0  c  g(n)  f(n)  n  n 0 l Asymptotic tight bound: n f(n) is  (g(n)) if  positive constants c 1, c 2, and n 0 such that c 1 g(n)  f(n)  c 2 g(n)  n  n 0 n f(n) =  (g(n)) if and only if f(n) = O(g(n)) AND f(n) =  (g(n))

Other Asymptotic Notations l A function f(n) is o(g(n)) if for any positive constant c,  n 0 such that f(n) < c g(n)  n  n 0 l A function f(n) is  (g(n)) if for any positive constant c,  n 0 such that c g(n) < f(n)  n  n 0 l Intuitively,  o() is like <  O() is like    () is like >   () is like    () is like =

l How comfortable are you with recursion?

Merge Sort MergeSort(A, left, right) { if (left < right) { mid = floor((left + right) / 2); MergeSort(A, left, mid); MergeSort(A, mid+1, right); Merge(A, left, mid, right); } // Merge() takes two sorted subarrays of A and // merges them into a single sorted subarray of A // (how long should this take?)

How Merge works

Merge Sort: Example Show MergeSort() running on the array A = {10, 5, 7, 6, 1, 4, 8, 3, 2, 9};

Neat little example from the book

Analysis of Merge Sort StatementEffort l So T(n) =  (1) when n = 1, and 2T(n/2) +  (n) when n > 1 l So what (more succinctly) is T(n)? MergeSort(A, left, right) {T(n) if (left < right) {  (1) mid = floor((left + right) / 2);  (1) MergeSort(A, left, mid);T(n/2) MergeSort(A, mid+1, right);T(n/2) Merge(A, left, mid, right);  (n) }

Recurrences l The expression: is a recurrence. n Recurrence: an equation that describes a function in terms of itself on smaller inputs

Recurrence Examples

Solving Recurrences l Substitution method l Iteration method l Master method

Solving Recurrences l The substitution method (CLR 4.1) n A.k.a. the “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  ??? Tree method often used

Solving Recurrences l The substitution method (CLR 4.1) n A.k.a. the “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  T(n) =  (n lg n) u T(n) = 2T(  n/2  + 17) + n  ???

Solving Recurrences l The substitution method (CLR 4.1) n A.k.a. the “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  T(n) =  (n lg n) u T(n) = 2T(  n/2  + 17) + n   (n lg n)

Substitution examples l T(n) = 2T(n/2) + kn, T(1)=c’ (e.g., merge sort) n Guess T(n) <= c.nlgn does not work n Guess T(n) =c’, c>=k l T(n) = 2T(n/2) + k, T(1)=c’ (e.g., D&C search) n Guess T(n) <= cn does not work n Guess T(n) =k, c>=d+c’ l T(n) = 2T(√n) + k.lg n n Change variable first Must prove the exact form guessed; otherwise revise your guess

Solving Recurrences l Another option is what the book calls the “iteration method” n Expand the recurrence n Work some algebra to express as a summation n Evaluate the summation l We will show several examples

l s(n) = c + s(n-1) c + c + s(n-2) 2c + s(n-2) 2c + c + s(n-3) 3c + s(n-3) … kc + s(n-k) = ck + s(n-k)

l So far for n >= k we have n s(n) = ck + s(n-k) l What if n=k? n s(n) = cn + s(0) = cn

l So far for n >= k we have n s(n) = ck + s(n-k) l What if k = n? n s(n) = cn + s(0) = cn l So l Thus in general n s(n) = cn

l s(n) =n + s(n-1) =n + n-1 + s(n-2) =n + n-1 + n-2 + s(n-3) =n + n-1 + n-2 + n-3 + s(n-4) =…=… = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

l s(n) =n + s(n-1) =n + n-1 + s(n-2) =n + n-1 + n-2 + s(n-3) =n + n-1 + n-2 + n-3 + s(n-4) =… = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k) =

l So far for n >= k we have

l What if k = n?

l So far for n >= k we have l What if k = n?

l So far for n >= k we have l What if k = n? l Thus in general

l T(n) = 2T(n/2) + c 2(2T(n/2/2) + c) + c 2 2 T(n/2 2 ) + 2c + c 2 2 (2T(n/2 2 /2) + c) + 3c 2 3 T(n/2 3 ) + 4c + 3c 2 3 T(n/2 3 ) + 7c 2 3 (2T(n/2 3 /2) + c) + 7c 2 4 T(n/2 4 ) + 15c … 2 k T(n/2 k ) + (2 k - 1)c

l So far for n > 2 k-1 we have n T(n) = 2 k T(n/2 k ) + (2 k - 1)c l What if n=2 k, i.e., k = lg n? n T(n) = 2 lg n T(n/2 lg n ) + (2 lg n - 1)c = n T(n/n) + (n - 1)c = n T(1) + (n-1)c = nc + (n-1)c = (2n - 1)c