1. DR. Gatot F. Hertono, MSc. Design and Analysis of ALGORITHM (Session 2)

2. Our Machine Model: Assumptions Generic Random Access Machine (RAM)  Executes operations sequentially  Set of basic operations: Arithmetic. Logical, Comparisons, Function calls  Simplifying assumption: all ops cost 1 unit Eliminates dependence on the speed of our computer, otherwise impossible to verify and to compare Notes:

3. Running time The running time depends on the input. Example: an already sorted sequence is easier to sort. Major Simplifying Convention: Parameterize the running time by the size of the input.  T A (n) = time of A on length n inputs Generally, we seek upper bounds on the running time, to have a guarantee of performance.

4. Examples of Basic Operations AlgorithmInput TypesBasic Operations List SearchingList with n elementsComparation List SortingList with n elementsComparation Matrix Productn x n matricesScalar Products Prime Factorisationn digit numbersScalar Division Polynomial Evaluationn degree polynomialScalar Products Tree TraversalTree with n nodesVisiting a node Notes: The running time of an algorithm is determined by its input size n

5. Time Complexity The complexity of an algorithm is determined by the number of basic operations and how many time the algorithm computes those basic operations. Notes: The complexity analysis is machine independent. Time complexity of an algorithm will determine the running time depends on its input size, i.e. the time complexity is a function of input size. Time Complexity maps “input size” to “time” T(n) executed.

6. Purpose  To estimate how long a program will run.  To estimate the largest input that can reasonably be given to the program.  To compare the efficiency of different algorithms.  To help focus on the parts of code that are executed the largest number of times.  To choose an algorithm for an application.

7. Time Complexity: an example

8. Best, Worst and Average Case Sometimes, given two different inputs with a same size, an algorithm can have different running time. Example: Suppose a sorting algorithm has some inputs with a same size but different order: -Input 1: 10, 5, 23, 45, 1, 100 -Input 2: 1,5,10, 23,45, 100 -Input 3: 100, 45, 23, 10, 5, 1 Do those inputs give the same running time? In ascending order Average case Best case Worst case

9. Best, Worst and Average Case (cont.)

10. Best, Worst and Average Cases (cont.) Best-case Complexity: is a function B(n)  B(n) = min{  (i)  i  I n } Let I n denote a set of all input with size n of an algorithm and  (i) denote the number of primitive operations of the corresponding algorithm when given input i. Worst-case Complexity: is a function W(n)  W(n) = max{  (i)  i  I n } Average-case Complexity: is a function A(n)  A(n) = where p(i) is the probability of i occurs as an input of an algorithm. Mathematically, we can define:

11. Example of insertion sort 824936

12. 824936

13. 824936 284936

14. 824936 284936

15. 824936 284936 248936

16. 824936 284936 248936

17. 824936 284936 248936 248936

18. 824936 284936 248936 248936

19. 824936 284936 248936 248936 234896

20. 824936 284936 248936 248936 234896

21. 824936 284936 248936 248936 234896 234689done

22. Running Time of Insertion Sort 1 Insertion_Sort(A) for j  2 to length(A) Cost c1 Times n 2 key  A(j) c2n-1 3 i  j-1 c3n-1 4 while i > 0 and A(i) > key c4 5 A(i+1)  A(i) c5 6 i  i – 1 c6 7 A(i+1)  key c7n-1 n = length(A) t j = number of while loop execution for a certain value j

23. Insertion Sort: an analysis Best case: in an ordered list (i.e t j = 1, for j = 2, …, n) Worst case: in a reverse ordered list (i.e t j = j, for j = 2, …, n)

24. Time Complexity: a comparison

25. Machine-independent time What is insertion sort’s worst-case time? B IG I DEAS : Ignore machine dependent constants, otherwise impossible to verify and to compare algorithms Look at growth of T(n) as n → ∞. “Asymptotic Analysis”

26. Analysis  Simplifications Ignore actual and abstract statement costs Order of growth is the interesting measure:  Highest-order term is what counts Remember, we are doing asymptotic analysis As the input size grows larger it is the high order term that dominates

27. Assignment 1  In order to show that an algorithm is not unique, design two different algorithms of a specific problem.  Design an algorithm and show its time complexity to compute a product of two n x n matrices (how is the time complexity in the best and worst cases?)

28. Performance & Speed