Computer Science CS 330: Algorithms Gene Itkis

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis2 Complexity: outline  Evaluating efficiency  What to count  Ignoring constants  Big-O, and other letters  “Direct” 1.Make an educated guess 2.Prove the guess (or adjust it and try again)  Recurrences  Same as above, or  Master Method

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis3 Performance/Complexity  Depends on input  Input size:  Express complexity as a function of input size  Examples: Selection Sort is O(n 2 ), Merge – O(n lg n)  Specific instance:  Best Case  Worst Case  Average  Costs of specific operations  Choose the right operations  Ignore some details: big-O notation

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis4 Performance/Complexity for i= 1 to n do min_index <-- i for j= i+1 to n do if A[j] < A[min_index] then min_index <-- j if A[j] < A[min_index] then min_index <-- j swap( A[i], A[min_index] ) n Depends on input size: n elements Best/Worst/Average Cases: here  same Cost of specific operations: Comparisons Assignments/moves Other (e.g. arithmetic)  typical  sorting crates

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis5 Cost of various operations  How to compare the costs of +, <, := ?  Usually they are “similar”?  For each <, we usually do a few of +, :=, etc.  The difference between different options is often  a constant factor  Big-O allows to hide these details  As we saw, these details are often secondary to “n vs n 2 ” and similar relations  Constant factors hidden in big-O are often similar between the various options  Discussion section!

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis6 Selection Sortfor i= 1 to n do min_index <-- i for j= i+1 to n do if A[j] < A[min_index] then min_index <-- j swap( A[i], A[min_index] ) nn×nn× 1 1? 3?  O(1) ? × O(1) ? = n “average”  n /2 iterations × O(1) nn O( n 2 )Total: n × n /2 × O(1) = O( n 2 ) nnn O( n 2 ) Total: [( n -1)+( n -2)+( n -3)+…+1] × O(1) = n(n- 1)/2=O( n 2 ) nn Recurrence: T( n )=T( n -1) + O(1)

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis7 Merge Sort Merge_Sort(A[], first, last): if (last < first) then return else middle   (first+last)/2  Merge_Sort( A, first, middle ) Merge_Sort( A, middle+1, last ) Merge( A[first...middle], A[middle+1...last], A[first...last] ) return A[] Divide & Concur Reduction

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis8 Recurrence Relations  Selection Sort  T(n) = T(n-1) + O(n)  Merge Sort  T(n) = 2T(n/2) + O(n) Reduction: Sel_Sort Sel_Sort(L): Rem_Min x  Rem_Min(L); Sel_Sort Sel_Sort(L); Ins_Head L  Ins_Head(x,L); return L M_S M_S(A[], f, l): … M_S M_S( A, f, mid ) M_S M_S( A, mid+1, l ) Merge Merge( A[f...mid], A[mid+1...l], A[f...l] )

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis9 Recurrence Relations  Binary Search Bin_Search Bin_Search( x, A[], f, l ): if (l < f) then return mid   (f+l)/2  case A[m]=xm, A[m] A[m]=x: return ( m, A[m] ) A[m]<xBin_Searchx,A, f, mid-1 A[m]<x: return Bin_Search( x,A, f, mid-1 ) A[m]>xBin_Searchx,A, mid+1, l A[m]>x: return Bin_Search( x,A, mid+1, l ) T(n) = T(n/2) + O(1)

Computer Science CS-330: Algorithms, Fall 2004Gene Itkis10 Common Recurrences  T(n) = T(n-1) + O(n)  T(n) = 2T(n/2) + O(n)  T(n) = T(n/2) + O(1)  T(n) = 2T(n-1) + O(n) O(n 2 ) O(n lg n) O( lg n ) O(2 n ) O( n )