Basic Concepts 2011, Fall Pusan National University Ki-Joune Li

STEMPNU 2 Data Structures ? Refrigerator Problem  If we have only one item in my refrigerator, no problem.  If we have several items in a refrigerator, How to place milk, eggs, kimchi, beers, etc. in a refrigerator ? How to measure the efficiency ? Placement and cooking  Placement : something about materials  Cooking : something about activities or processes  Some Organization or Structures Data Structures  How to place data in memory

STEMPNU 3 What is good data structure? What is good placement in refrigerator ?  Easy to cook (and place) What is good data structure ?  Easy to process  Data Structures (Placement of materials) + Algorithms (Cooking) = Program (Meal)  Good Data Structures support good algorithms Method to process : Algorithm

STEMPNU What are the core functions of data structures? 4

STEMPNU 5 Abstract Data Type Object-Oriented Programming  Object ? Abstraction (or Encapsulation)  Hiding the internal details Implementation Internal mechanism and process  Only provide Interfaces Abstract Data Type  Hiding the internal structures once it has been implemented  Provide only the interface to the users

STEMPNU 6 Example of Abstract Data Types Example ADT ADT NaturalNumber is functions: for all x, y in NaturalNumber; true, false in Boolean, and where +,-,<,==, and = are integer operations Zero(): NaturalNumber ::= 0 IsZero(x): Boolean ::= if(x==0) isZero=true else isZero=false Add(x,y): NaturalNumber ::= if(x+y<=MAXINT) Add=x+y else ADD=MAXINT End NaturalNumber Implementation Zero IsZero Add

STEMPNU 7 Algorithms Algorithm  A sequence of instructions with specifications of Input Output : at least one output Definiteness : Clear instructions Finiteness Effectiveness  Abstract description of a program Can be easily converted to a program Correctness of algorithm

STEMPNU 8 Performance Analysis What is a good algorithm ?  Correctness  Good documentation and readable code  Proper structure  Effective How to measure the effectiveness of an algorithm ?  Space complexity Amount of memory it needs to run  Time complexity Amount of time (mostly CPU time) it needs to run

STEMPNU 9 Space Complexity Notation  Space complexity, f (n) : function of input size n  How should the constants be determined ?  Is it meaningful to count the number of bytes ? int sumAB(int a, int b) { int sum; sum=a+b; return sum; } int sumArray(int n, int *a) { int sum=0; for(int i=0;i<n;i++) sum += a[i]; return sum; } int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); } f (n) = c 1 c 1 : constant f (n) = c 2 a[n] ? f (n) = c 3 n why ?

STEMPNU 10 Time Complexity Notation  Time complexity, f (n) : function of input size n int sumAB(int a, int b) { int sum; sum=a+b; return sum; } int sumArray(int n, int *a) { int sum=0; for(int i=0;i<n;i++) sum += a[i]; return sum; } int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); } f (n) = 2 +  f (n) = 3 n +1 +  f (n) = (2+  ) n Is it really meaningful to determine these constants ?

STEMPNU 11 Asymptotic Notation Exact time (or step count) to run  Depends on machines and implementation (program)  NOT good measure More general (but inexact) notation : Asymptotic Notation  Big-O O(n) : Upper Bound  Omega-O  (n) : Lower Bound  Theta-O  (n) : More precise than Big-O and Omega-O  Big-O notation is the most popular one.

STEMPNU 12 Big-O notation Definition ( f of n is big-O of g of n)  f (n)  O(g (n))  there exist c and n 0 (c, n 0 >0) such that f (n)  c·g(n), for all n (  n 0 ) Example  3n + 2 = O(n), 3n + 2 = O(n 2 )  Time complexity of the following algorithm f (n)= O(n) int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); }

STEMPNU 13 Big-O notation : Some Properties Classification  O(1): constant, O(n): Linear, O(n 2 ): Quadratic, O(2 n ): exponential Polynomial function  If f (n) = a m n m + a m-1 n m-1 + … + a 1 n + a 0, then f (n)  O(n m ) Big-O is determined by the highest order Only the term of the highest order is of our concern Big-O : useful for determining the upper bound of time complexity  When only an upper bound is known, Big-O notation is useful  In most cases, not easy to find the exact f (n)  Big-O notation is the most used.

STEMPNU 14 Omega-O notation Definition ( f of n is Omega-O of g of n)  f (n)   (g (n))  there exist c and n 0 (c, n 0 >0) such that f (n)  c·g(n), for all n (  n 0 ) Example  3n + 2 =  (n), 3n + 2 =  (1), 3n =  (n),  Time complexity of the following algorithm f (n)=  (n) If f (n) = a m n m + a m-1 n m-1 +…+ a 1 n+a 0, then f (n)   (n m )  Omega-O is determined by the highest order Omega-O notation : useful to describe the lower bound int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); }

STEMPNU 15 Theta-O notation Definition ( f of n is theta-O of g of n)  f (n)   (g (n))  there exist c 1, c 2 and n 0 (c 1, c 2, and n 0 >0) such that c 1 ·g(n)  f (n)  c 2 ·g(n), for all n (  n 0 ) Example  3n + 2 =  (n), 3n + 2   (1), 3n   (n),  Time complexity of the following algorithm f (n)=  (n) If f (n) = a m n m + a m-1 n m-1 +…+ a 1 n+a 0, then f (n)   (n m )  Theta-O is determined by the highest order Theta-O  Possible Only if f (n)=  (g(n)), and f(n)=  (g(n))  Lower bound and Upper bound is the same  very exact but not easy to find such a g(n) int sumArray(int n, int *a) { if(n<=0) return 0; return a[n-1]+sumArrary(n-1,a); }

STEMPNU 16 Complexity Analysis Worst-Case Analysis  Time complexity for the worst case  Example Linear Search : f (n) = n Average Analysis  Average time complexity  f (n) = p 1 f 1 (n) + p 2 f 2 (n) + … p k f k (n), where p i is the probability for the i -th case.  Not easy to find p i In most cases, only worst-case analysis Why not Best-Case Analysis ?

STEMPNU 17 Example : Worst-Case Time complexity of Binary Search Binary Search  Big-O : O(n 2 ), O(log n)  Omega O :  (1),  (log n)  Theta O :  (log n) int BinarySearch(int v, int *a, int lower, int upper) // search m among a sorted array a[1ower], a[lower+1], … a[upper] { if(l<=u) { int m=(lower+upper)/2; if(v>a[m]), return BinarySearch(v,a,m+1,upper); else if(v<a[m]) return BinarySearch(v,a,lower,m-1); else /* v==a[m] */ return m; } return -1; // not found }

STEMPNU 18 Comparison : O(1), O(log n), O(n), O(n 2 ), O(2 n ) Graph In general  Algorithm of O(1) is almost impossible  Algorithms of O(log n) is excellent  Algorithms of O(n) is very good  Algorithms of O(n 2 ) is not bad  Algorithms of O(n 3 ) is acceptable  Algorithms of O(2 n ) is not useful