Algorithm Efficiency and Sorting Data Structure & Algorithm

2 Measuring the Efficiency of Algorithms Analysis of algorithms –Provides tools for contrasting the efficiency of different methods of solution Time efficiency, space efficiency

3 The Execution Time of Algorithms Counting an algorithm's operations is a way to assess its time efficiency –An algorithm’s execution time is related to the number of operations it requires –Example: Traversal of a linked list of n nodes need n steps for reading the nodes or writing to the output screen –Example: For loop with n data?

4 Algorithm Growth Rates An algorithm’s time requirements can be measured as a function of the problem size –Number of nodes in a linked list –Size of an array –Number of items in a stack Algorithm efficiency is typically a concern for large problems only

5 Algorithm Growth Rates Figure 9-1 Time requirements as a function of the problem size n Algorithm A requires time proportional to n 2 Algorithm B requires time proportional to n

6 Algorithm Growth Rates An algorithm’s growth rate –Enables the comparison of one algorithm with another –Algorithm A requires time proportional to n 2 –Algorithm B requires time proportional to n –Algorithm B is faster than Algorithm A –n 2 and n are growth-rate functions –Algorithm A is O(n 2 ) - order n 2 –Algorithm B is O(n) - order n Big O notation Also called complexity time

7 Big O Notation Big ‘O’ notation is denoted as O(acc) O - order acc - class of algorithm complexity that may consist of 1, log x n, n, n log x n, n 2, …

8 Order-of-Magnitude Analysis and Big O Notation Order of growth of some common functions –O(1) < O(log 2 n) < O(n) < O(n * log 2 n) < O(n 2 ) < O(n 3 ) < O(2 n ) Properties of growth-rate functions –O(n 3 + 3n) is O(n 3 ): ignore low-order terms –O(5 f(n)) = O(f(n)): ignore multiplicative constant in the high-order term

9 Order-of-Magnitude Analysis and Big O Notation Worst-case analysis –A determination of the maximum amount of time that an algorithm requires to solve problems of size n Average-case analysis –A determination of the average amount of time that an algorithm requires to solve problems of size n Best-case analysis –A determination of the minimum amount of time that an algorithm requires to solve problems of size n

10 Keeping Your Perspective Frequency of operations –When choosing an ADT’s implementation, consider how frequently particular ADT operations occur in a given application Bad big O: –Frequent use - but smaller data – OK! –Frequent use – bigger amount of data – Not OK! –Bigger amount of data, not frequent use – OK!

11 Keeping Your Perspective If the problem size is always small, you can probably ignore an algorithm’s efficiency –Order-of-magnitude analysis focuses on large problems Weigh the trade-offs between an algorithm’s time requirements and its memory requirements Compare algorithms for both style and efficiency

12 Big O Notation NotationExecution time / number of step O(1)Constant. Independent of the input size, n. O(log x n)Logarithmic increase.

13 Big O Notation O(n)Linear increase. Increase directly with the input size, n. O(n log x n)log-linear increase O(n 2 )Quadratic increase. Practical for average input size, n. O(n 3 )Cubic increase. Practical for small input size, n. O(2 n )Exponential increase. Increase too rapidly to be practical

14 Big O Notation

15 Big O Notation

16 Determine the complexity time of algorithm can be determined - theoretically – by calculation - practically – by experiment /implementation

17 Determine the complexity time of algorithm - practically –Implement the algorithms in any programming language and run the programs –Depend on the compiler, computer, data input and programming style.

18 Determine the complexity time of algorithm - theoretically The complexity time is related to the number of steps /operations. Complexity time can be determined by 1. Count the number of steps and then find the class of complexity. Or 2. Find the complexity time for each steps and then count the total.

19 Determine the number of steps The following algorithm is categorized as O(n). int counter = 1; int i = 0; for (i = 1; i <= n; i++) { cout << "Arahan cout kali ke " << counter << "\n"; counter++; }

20 Determine the number of steps Numstatements 1 int counter = 1; 2 int i = 0; 3 i = 1 4 i <= n 5 i++ 6 cout << "Arahan cout kali ke " << counter << "\n" 7 counter++

21 Statement 3, 4 & 5 are the loop control and can be assumed as one statement. NumStatements 1 int counter = 1; 2 int i = 0; 3 i = 1; i <= n; i++ 6 cout << "Arahan cout kali ke " << counter << "\n" 7 counter++ Determine the number of steps

22 statement 3, 6 & 7 are in the repetition structure. It can be expressed by summation series  = f(1) + f(2) f(n) = n i = 1 n f(i) f(i) – statement executed in the loop Determine the number of steps- summation series

23 example:- if n = 5, i = 1 The statement that represented by f(i) will be repeated 5 time = n times = O (n)  = f(1) + f(2) + f(3) + f(4) + f(5) = 5 i = 1 5 f(i) Determine the number of steps- summation series

24 Determine the number of steps- summation series statementsNumber of steps int counter = 1; int i = 0; i = 1; i= n; i++ cout << "Arahan cout kali ke " << counter << "\n" counter++  = 1 i=1 1 f(i)  = 1 i=1 1 f(i)  = n i=1 n f(i) . i=1 n f(i)  = n. 1 = n i=1 1 f(i) . i=1 n f(i)  = n. 1 = n i=1 1 f(i)

25 Determine the number of steps- summation series Total steps: n + n + n = 2 + 3n Consider the largest factor. Algorithm complexity can be categorized as O(n)

26 Determine the number of steps- through actual counting Line num Statement Total steps start int counter = 1; int i = 0 for (i = 1; i <= n; i++) cout << "Arahan cout kali ke " << counter << "\n"; counter++; End - 1 n n.1 = n -

27 Determine the number of steps- through actual counting Statements 1 and 2  executed only once, 0 (1) Statement 3  linear execution, 0 (n) Statements 4, 5, 6  executed only once, 0 (1) HOWEVER because those statements is in a for loop, execution is linear, 0 (n) Algorithm complexity time:- T(n) = n + n + n = 2 + 3n = O (n)

28 Shortcut formula for loop total steps Total Steps = b-a+l b = loop final conditional value a = loop initial conditional value l = constant value for loop increment Example – using the statement 3 (slide 38 th ) b = n a = 1 l = 1 Total steps = b – a + 1 = n – = n

29 PenyataanBilangan langkah void contoh1 ( ) { cout << “ Contoh kira langkah “; } Jumlah langkah1  Jum bil langkah =1(nilai malar) ; masa kerumitan = O (1) PenyataanBilangan langkah void contoh2 ( ) { for (int a=1; a<=5; a++) cout << “ Contoh kira langkah “; } =5 5.1=5 0 Jumlah langkah10

30 PenyataanBilangan langkah void contoh3 ( ) { for (int a=1; a<=n; a++) cout << “ Contoh kira langkah “; } 0 n-1+1=n n.1=n 0 Jumlah langkah2n  Jum bil langkah =10(nilai malar) ; masa kerumitan = O (1) Samb…  Jum bil langkah =2n(nilai yg b’gantung pd n) ; masa kerumitan = O (n)

31 PenyataanBilangan langkah void contoh4 ( ) { for (int a=2; a<=n; a++) cout << “ Contoh kira langkah “; } 0 n-2+1=n-1 (n-1).1=n-1 0 Jumlah langkah2(n-1)  Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O (n) PenyataanBilangan langkah void contoh5 ( ) { for (int a=1; a<=n-1; a++) cout << “ Contoh kira langkah “; } 0 n-1-1+1=n-1 (n-1).1=n-1 0 Jumlah langkah2(n-1)

32 PenyataanBilangan langkah void contoh6 ( ) { for (int a=1; a<=n; a++) for (int b=1; b<=n; b++) cout << “ Contoh kira langkah “; } 0 n-1+1=n n.(n-1+1)=n.n n.n.1=n.n 0 Jumlah langkahn+2n 2 Samb…  Jum bil langkah =n+2n 2 (nilai yg b’gantung pd n 2 ) ; masa kerumitan = O (n 2 )  Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O (n)

33 Count the number of steps and find the Big ‘O’ notation for the following algorithm int counter = 1; int i = 0; int j = 1; for (i = 3; i <= n; i = i * 3) { while (j <= n) { cout << "Arahan cout kali ke " << counter << "\n"; counter++; j++; } Determine the number of steps - exercise

34 statementsNumber of steps int counter = 1; int i = 0; int j = 1; i = 3; i <= n; i = i * 3 j <= n  = 1 i=1 1 f(i)  = 1 i=1 1 f(i)  = 1 i=1 1 f(i)  = f(3) + f(9) + f(27) + … + f(n) = log 3 n i=3 n f(i) . i=3 n f(i)  = log 3 n. n j=1 n f(i) Determine the number of steps - solution

35 cout << "Arahan cout kali ke " << counter << "\n"; counter++; j++; . i=3 n f(i) . = log 3 n. n. 1 j=1 n f(i)  i=1 1 f(i) . i=3 n f(i) . = log 3 n. n. 1 j=1 n f(i)  i=1 1 f(i) Determine the number of steps - solution . i=3 n f(i) . = log 3 n. n. 1 j=1 n f(i)  i=1 1 f(i)

36 => log 3 n + log 3 n. n + log 3 n. n. 1 + log 3 n. n. 1 + log 3 n. n. 1 => 3 + log 3 n + log 3 n. n + log 3 n. n + log 3 n. n + log 3 n. n => 3 + log 3 n + 4n log 3 n Total steps: Determine the number of steps - solution

log 3 n + 4n log 3 n Consider the largest factor (4n log 3 n) and remove the coefficient (n log 3 n) In asymptotic classification, the base of the log can be omitted as shown in this formula: log a n = log b n / log b a Thus, log 3 n = log 2 n / log 2 3 = log 2 n / 1.58… Remove the coefficient 1/ So we get the complexity time of the algorithm is O(n log 2 n) Determine the number of steps - solution