1. Program Performance 황승원 Fall 2010 CSE, POSTECH

2. Publishing Hwang’s Algorithm Hwang’s took only 0.1 sec for DATASET1 in her PC while Dijkstra’s took 0.2 sec. TWO TIMES FASTER!!!! (maybe I deserve two T awards) Only for this set? (best, worst, average) Scalable to problem size? Only in her computer (specialized for some operation?)

3. Program Performance Program performance is the amount of computer memory and time needed to run a program. How is it determined? – Analytically performance analysis – Experimentally performance measurement

4. Criteria for Measurement Space – amount of memory program occupies – usually measured in bytes or MB Time – execution time – usually measured by the number of executions

5. Space Complexity Space complexity is defined as the amount of memory a program needs to run to completion. Why is this of concern? – We may not have sufficient memory on our computer. – The space complexity may define an upper bound on the data that the program can handle. (e.g., 8,168,684,336 pages, Aug 2005)

6. Components of Program Space Program space = Instruction space + data space + stack space The instruction space is dependent on several factors. – the compiler that generated the machine code – the compiler options that were set at compilation time – the target computer

7. Components of Program Space Data space – very much dependent on the computer architecture and compiler – The magnitude of the data that a program works with is another factor char1float4 short2double8 int2long double10 long4pointer2 Unit: bytesSee Figure 2.2

8. Components of Program Space Data space – Choosing a “smaller” data type has an effect on the overall space usage of the program. – Choosing the correct type is especially important when working with arrays. – How much memory is allocated with these declarations? double a[100]; int a[100]; x4!!

9. Components of Program Space Environment Stack Space – Every time a function is called, the following data are saved on the stack. 1. the return address 2. the values of all local variables and value formal parameters 3. the binding of all reference and const reference parameters – What is the impact of recursive function calls on the environment stack space? e.g., fact(n)=n*fact(n-1), fact(1)=1 fact(2) =fact(2)*3 =(1*2)*3 =(fact(1)*2)*3 fact(1) fact(3)

10. Space Complexity Summary To make your programs more space efficient? – Always choose the optimal (smallest necessary) data type – Choose non-recursive algorithms when appropriate. READ: Chapter 2.2.2

11. Time Complexity Time complexity is the amount of computer time a program needs to run. Why do we care about time complexity? – Some programs require a real-time response. – If there are many solutions to a problem, typically we’d like to choose the quickest.

12. Time Complexity How do we measure? – Count a particular operation(operation counts) e.g., comparisons in sorting – Count the number of steps(step counts) – Asymptotic complexity

13. Human Experiment: SORT!

14. Insertion sort (hand simulation) 31524 [0] [1] [2] [3] [4] 13524 ij t=1 Best case? Worst case?

15. Running Example: Insertion Sort for (int i = 1; i < n; i++) // n is the number of // elements in array { // insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; } 3122 n=3 i=1 1322 i=2 1223 j=0

16. Comparison Count for (int i = 1; i < n; i++) { // insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t; }

17. Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; How many comparisons are made? The number of compares depends on a[]s and t as well as on n.

18. Comparison Count Worst case count = maximum count Best case count = minimum count Average count

19. Worst Case Comparison Count for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a = [1,2,3,4] and t = 0 ⇒ 4 compares a = [1,2,3,4,…,i] and t = 0 ⇒ i compares

20. Worst Case Comparison Count for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; total compares = 1+2+3+…+(n-1) = (n-1)n/2 READ Examples 2.7~2.18

21. Step Count The operation-count method omits accounting for the time spent on all but the chosen operation The step-count method count for all the time spent in all parts of the program A program step is loosely defined to be a syntactically or semantically meaningful segment of a program for which the execution time is independent of the instance characteristics. – 100 adds, 100 subtracts, 1000 multiples can be counted as one step. – However, n adds cannot be counted as one step. EXACT COUNT DOESN’T MEAN MUCH

22. Step Count steps/execution (s/e) for (int i = 1; i < n; i++)1 { 0 // insert a[i] into a[0:i-1]0 int t = a[i];1 int j;0 for (j = i - 1; j >= 0 && t < a[j]; j--)1 a[j + 1] = a[j];1 a[j + 1] = t;1 } 0

23. Step Count s/efrequency for (int i = 1; i < n; i++)1 n-1 { 0 0 // insert a[i] into a[0:i-1]0 0 int t = a[i];1 n-1 int j;0 0 for (j = i - 1; j >= 0 && t < a[j]; j--)1 (n-1)n/2 a[j + 1] = a[j];1 (n-1)n/2 a[j + 1] = t;1 n-1 } 0 n-1

24. Step Count Total step counts = (n-1) + 0 + 0 + (n-1) + 0 + (n-1)n/2 + (n-1)n/2 + (n-1) + (n-1) = n 2 + 3n – 4 READ Examples 2.19~2.23

25. Asymptotic Complexity Neither of the two yield a very accurate measure – Operation counts: focus on “key” operations and ignore all others – Step counts: the notion of a step is itself inexact Asymptotic complexity provides meaningful statements about the time and space complexities of a program

26. Asymptotic Notation Describes the behavior of the time or space complexity for large instance characteristic Big Oh (O) notation provides an upper bound for the growth rate of the given function Omega (Ω) notation provides a lower bound Theta (  ) notation is used when an algorithm can be bounded both from above and below by the same function

27. Big Oh Time complexity T(n) of Insertion sort lies in the complexity class O(n 2 ). There exists c and n 0 such that time or number of operations does not exceed c.n 2 on any input of size n (n >= n 0 ). e.g., c=3 n 0 =1

28. Omega Time complexity T(n) of Insertion sort lies in the complexity class Ω(n 2 ). There exists c and n 0 such that time or number of operations exceeds c.n 2 on any input of size n (n >= n 0 ). e.g., c=1 n 0 =2

29. Theta Examples As seen above the lower bound for the insertion sort problem lies in  (n 2 ). Its upper bound lies in O(n 2 ) Thus,  (n 2 ) How good is it?

30. Smart Lingo: Common Growth Rate Functions 1 (constant): growth is independent of the problem size n. log 2 N (logarithmic): growth increases slowly compared to the problem size (good search) N (linear): directly proportional to the size of the problem. N * log 2 N (n log n): typical of some divide and conquer approaches (good sort) N 2 (quadratic): typical in nested loops N 3 (cubic): more nested loops 2 N (exponential): growth is extremely rapid and possibly impractical.

31. Beating Dummies: Practical Complexities Overly complex programs may not be practical given the computing power of the system. lognnnlognn2n2 n3n3 2n2n 012345012345 1 2 4 8 16 32 0 2 8 24 64 160 1 4 16 64 256 1024 1 8 64 512 4096 32768 2 4 16 256 65536 4294967296

32. O(n) Sort? Sort as good as search? Really? What does Wikipedia say?

33. Example of LSD-Radix Sort Input is an array of 15 integers. For integers, the number of buckets is 10, from 0 to 9. The first pass distributes the keys into buckets by the least significant digit (LSD). When the first pass is done, we have the following. 0 1 2 3 4 5 6 7 8 9

34. Example of LSD-Radix Sort…contd We collect these, keeping their relative order: Now we distribute by the next most significant digit, which is the highest digit in our example, and we get the following. When we collect them, they are in order. 0 1 2 3 4 5 6 7 8 9

35. Sort in search engine? Top-20? Special lectures – next two weeks

36. Coming Up Next READING: Ch 2-3 NEXT: Linear List– Array Representation (Ch 5)