CSC 212 – Data Structures Lecture 15: Big-Oh Notation

Question of the Day Use the numbers two, three, four, and five, one addition operator and one equality operator  Write mathematical equation (e.g., not in Java) so equality operator holds = 3 2

Analysis Techniques Running time is important, … … but cannot always compare  Lots of ways to solve a single problem  Solutions have lots of implementations

Pseudo-Code Only for human eyes  Ignore unimportant & implementation details  Instead use "pseudo-code" Pseudo-code isn't real  Used for outlining, designing, & analysis  No formal definition or “proper” writing  Use a language-like manner

Pseudo-Code Include important details  Loops, assignments, method calls, etc.  Helps better analyze algorithm Remember only to understand algorithm  Ignore punctuation and formalisms  Written to allow people to understand and analyze

Pseudo-code Example int factorial(n, n  Z + ) returnVariable  1 while (n > 0) returnVariable  returnVariable * n n  n – 1 return returnVariable

Big-Oh Notation Computes code complexity  Worst-case analysis of performance  Related to total execution time Used to compare approaches  Only requires algorithms  Not need to implement everything  Avoids unrelated details E.g., Compiler, CPU, users’ typing speed

Algorithmic Analysis

Algorithm Analysis Approximate time to run a program with n inputs on 1GHz machine: n = 10n = 50n = 100n = 1000n = 10 6 O(n log n)35 ns200 ns700 ns10000 ns20 ms O(n 2 )100 ns2500 ns10000 ns1 ms17 min O(n 5 )0.1 ms0.3 s10.8 s11.6 days3x10 13 years O(2 n )1000 ns13 days4 x years Too long! O(n!)4 ms Too long!

Big-Oh Notation Want results for large data sets  Worst case is 2 minutes -- nobody cares  Only consider major details Ignore multipliers  So, O(5n) = O(2n) = O(n)  Multipliers usually implementation-specific Ignore lesser terms  So, O(n 5 + n 2 ) = O(n 5 )  Does 17 minutes matter after 3x10 13 years?

What is n ? Analysis in respect to size of input  Question is what is size of input? Quick rules of thumb:  Analyze values below x  n = x  Analyze data in an array  n = size of array  Analyze linked list  n = size of linked list  Analyze 2 arrays  n = sum of array sizes

Analyzing an Algorithm Counts primitive operations executed  Assignments  Method calls  Performing arithmetic operation  Comparing two values  Indexing into array  Following a reference  Returning a method

Primitive Statements Run in constant time: O(1)  Fastest time possible Sequences also run in constant time  True if sequence not impacted by input  O(5) = O(5 * 1) = O(1)  Big-Oh is rough estimate – ignore constant multipliers

Simple Loops for (int i = 0; i < n.length; i++) { } while (i < n) { i++; }  Each loop executed n times  Loops only contain primitive statements  Total complexity of each loop is: O(n)

More Complicated Loops for (int i = 0; i < n; i += 2) { }  i assigned 0, 2, 4, 6,... until larger than n  Loop executes n / 2 times  Loop only contains primitive statements  Total complexity of loop is: O(n/2) = O(½ * n) = O(n)

Even More Complicated Loops for (int i = 1; i < n; i *= 2) { }  i assigned 1, 2, 4, 8,... until larger than n  Loop executes log 2 n times  Loop only contains primitive statements  Total complexity of loop is: O(log 2 n ) = O(log n)

Even More Complicated Loops for (int i = 1; i < n; i *= 3) { }  i assigned 1, 3, 9, 27,... until larger than n  Loop executes log 3 n times  Loop only contains primitive statements  Total complexity of loop is: O(log 3 n ) = O(log 3/log 2 * log n) = O(log n)

Loop Time Complexity When loop control variable increases:  Does not change: takes O(1) or O(∞) time  By constant value: takes O(n) time  By constant multiple: takes O(log n) time

Nested Loops for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { } }  Inner loop ( j ) executes in O(n) time  Outer loop ( i ) executes n times But each pass takes O(n) time, not O(1) time  Total complexity of nested loops: O(n) * O(n) = O(n 2 )

Your Turn Get back into groups and do activity

Before Next Lecture… Start week #6 assignment Continue lab #5 Programming assignment #2 next week