LECTURE 22: BIG-OH COMPLEXITY CSC 212 – Data Structures

Analysis Techniques  Running time is important, …  … but comparing times impossible in many cases  Single problem may have lots of ways to be solved  Many implementations possible for a solution

Pseudo-Code  Only for human eyes  Unimportant & implementation details ignored  Serves very real purpose, but language not real  Used when outlining, designing, & analyzing  Language-like manner to system, though not formal

Pseudo-Code  Include important details  Loops, assignments, method calls, etc.  Anything that you find helpful analyzing algorithm  Understanding algorithm is only goal  Feel free to ignore punctuation & formalisms  Allow people to understand and analyze; nothing else

Pseudo-code Example Algorithm factorial(n) returnVariable  1 while (n > 0) returnVariable  returnVariable * n n  n – 1 endwhile return returnVariable

Big-Oh Notation  Expresses an algorithm’s complexity  Worst-case analysis of performance  Usually closely correlated with execution time  Approaches compared using big-Oh notation  Requires algorithms only  Saves time implementing all the algorithms  Avoids unrelated details: CPU, typing speed, compiler

Algorithmic Analysis

Algorithm Analysis  Execution time with n inputs on 4GHz machine: n = 10n = 50n = 100n = 1000n = 10 6 O(n log n)9 ns50 ns175 ns2500 ns5 ms O(n 2 )25 ns625 ns2250 ns ns4 min O(n 5 )25000 ns72.5 ms2.7 s2.9 days1x10 13 yrs O(2 n )2500 ns3.25 days1 x yrs1 x yrs Too long! O(n!)1 ms1.4 x yrs7 x yrs Too long!

 Want results for large data sets  Nobody cares about 2 minute worst case run  Limit considerations to only the major details  Ignore multipliers  So, O (5 n ) = O (2 n ) = O ( n )  Multipliers usually implementation-specific  Does extra 20ms matter versus 4 minutes?  Ignore lesser terms  So, O ( n 5 + n 2 ) = O ( n 5 )  Does 17 minutes matter after 3x10 13 years? Big-Oh Notation

What is n ?  Big-Oh analysis always relative to input size  But determining input size is not always clear  Quick rules of thumb:  Analyze values below x : n = x  Analyze data in an array: n = size of array  Analyze linked list: n = size of linked list  Analyze 2 arrays: n = sum of array sizes

 Big-Oh counts primitive operations executed  Primitive operations include:  Assignments  Method calls  Performing arithmetic operation  Comparing two values  Getting entry from an array  Following a reference  Returning a value from a method Analyzing an Algorithm

Primitive Statements  Execute in constant time: O (1)  Fastest possible time  Sequence of primitives also runs in constant time  So long as input does not affect sequence  Constant multipliers ignored; only large sets matter  O (5) = O (5 * 1) = O (1)

for (int i = 0; i < n.length; i++){} -or- while (i < n) { i++; }  Each loop executed n times  Primitive statements only within the loop  Big-Oh complexity of each of these loops is: = O ( n ) Simple Loops

for (int i = 0; i < n.length; i++){} int i = 0; while (i < n) { i++; }  Each loop executed n times  Add complexities of sequential code  Big-Oh complexity of this code is: = O ( n ) + O ( n ) = O (2 n ) = O ( n ) Sequential Loops Add

for (int i = 0; i < n; i += 2) { } So, i  0, 2, 4, 6,..., n  Loop executes n / 2 times  Primitive statements only within the loop  Total big-Oh complexity of loop is: = O ( n / 2 ) = O (½ * n ) = O ( n ) More Complicated Loops

for (int i = 1; i < n; i *= 2) { } So, i  1, 2, 4, 8,..., n  Loop executes log 2 n times  Primitive statements only within the loop  Total big-Oh complexity of loop is: = O (log 2 n ) Really Complicated Loops

for (int i = 1; i < n; i *= 3) { } So, i  1, 3, 9, 27,..., n  Loop executes log 3 n times  Primitive statements only within the loop  Total big-Oh complexity of loop is: = O (log 3 n ) = O ( log 3 / log 2 * log n ) = O (log n ) Really Complicated Loops

Big-Oh Loop Complexity  Based upon how loop control variable increases:  Does not change: O (1) or O (∞) time  Increase by constant value: O ( n ) time  Constant multiple increase: O (log n ) time

for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { } }  Inner loop ( j ) executes in O ( n ) time  n times executing outer loop ( i )  But outer loop pass takes O ( n ) time  Total big-Oh complexity of these nested loops: = O ( n ) * O ( n ) = O ( n 2 ) Nested Loops Multiply

 Continue week #8 assignment  Due by 5PM next Tuesday  Examine programming assignment #3  Messages are not always sent to everyone!  Read section 4.3 in book before class  How do we prove big-Oh notations? Before Next Lecture…