1. Prof. Hsin-Mu (Michael) Tsai ( 蔡欣穆 ) Department of Computer Science and Information Engineering National Taiwan University BASICS

2. WHAT IS AN ALGORITHM? A computable set of steps to achieve a desired result. All algorithm must satisfy the following criteria: Input Output Definiteness Finiteness Effectiveness DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 2

3. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 3

4. WHAT IS A DATA STRUCTURE? An organization of information, usually in memory, for better algorithm efficiency. Or, a way to store and organize data in order to facilitate access and modifications. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 4

5. ALGORITHM SPECIFICATION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 5

6. HOW DO WE DESCRIBE AN ALGORITHM? Human language (English, Chinese, …) Programming language A mix of the above DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 6

7. EXAMPLE: SELECTION SORT DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 7 ㄅ 1 1 ㄆㄅ 2 1 ㄆㄅ 2

8. EXAMPLE: SELECTION SORT First attempt: for (i=0; i<n; ++i) { Examine list[i] to list[n-1] and suppose that the smallest integer is at list[min]; Interchange list[i] and list[min]; } Task 1 Task 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 8

9. void swap(int *x, int *y) { int temp = *x; *x=*y; *y=temp; } Or #define SWAP(x,y,t) ((t)=(x), (x)=(y), (y)=(t)) Task 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 9

10. min=i; for(j=i;j<n;++j) if (list[j]<list[min]) min=j; Task 1 Finally, see program 1.4 on p. 11 for a complete program of the selection sort we just develop. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 10

11. HOW DO WE PROVE THAT IT IS CORRECT? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 11

12. EXAMPLE: BINARY SEARCH DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 12

13. EXAMPLE: 1344 6 71113 1819 01234567891011 searchnum=13; return middle; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 13

14. EXAMPLE: 1344 6 71113 1819 0123456789 searchnum=13; left right middl e middle=(left+right)/2; left=middle+1; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 14 1011

15. EXAMPLE: 1344 6 71113 1819 0123456789 searchnum=5; return -1; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 15 1011

16. int binsearch(int list[], int searchnum, int left, int right) { int middle; while(left<=right) { middle=(left+right)/2; switch(COMPARE(list[middle], searchnum)) { case -1: left=middle+1; break; case 0: return middle; case 1: right=middle-1; } return -1; } DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 16

17. RECURSIVE ALGORITHMS What does “recursive” mean? A function which calls itself (direct recursion), or a function which calls other functions which call the calling function again (indirect recursion). Any function that we can write using assignment, if-else, and while statements can be written recursively. Why do we want to use recursive functions (or algorithms)? It allow us to express an otherwise complex process in very clear terms. Often the recursive function is easier to understand than its iterative counterpart. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 17

18. THE STORY OF RECURSIVE FUNCTION Function blah Function blah (clone 1) Function blah (clone 2) Function blah (clone 2) 1 1 2 2 3 3 ㄅ is too much for me. I’ll partition it, do part 1, and clone two copies of myself to do the rest. Yo. Do the work ㄅ. I’m ㄅ. The work to be done. Hey man. Do the work ㄅ part 2. Hi. Do the work ㄅ part 3. ㄅ ㄅ DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 18

19. EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 19

20. EXAMPLE 2 int binsearch(int list[], int searchnum, int left, int right) { int middle; if (left<=right) { middle=(left+right)/2; switch (COMPARE(list[middle], searchnum)) { case -1: return binsearch(list, searchnum, middle+1, right); case 0: return middle; case 1: return binsearch(list, searchnum, left, middle-1); } 1. Termination condition 2. Recursive calls DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 20

21. EXAMPLE 3 Output all permutations Input: {a,b,c} Output: (a,b,c),(a,c,b),(b,a,c),(b,c,a),(c,a,b),(c,b,a) How do we write this program recursively? Example: input={a,b,c,d}. Output = a followed by all permutations of {b,c,d} b followed by all permutations of {a,c,d} c followed by all permutations of {a,b,d} d followed by all permutations of {a,b,c} “all permutations of {…}”  recursive calls! Homework: read and understand program 1.9 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 21

22. DATA ABSTRACTION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 22

23. DATA TYPE What is a data type? A data type is a collection of objects and a set of operations that act on those objects. Data types in C char, int, float, long, double (unsigned, signed, …) Array Struct struct { int a; int b; char str[16]; int * iptr; } blah; int iarray[16]; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 23

24. DATA TYPE Operations +, -, *, /, %, == =, +=, -= ? : sizeof, - (negative) giligulu(int a, int b) DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 24

25. DATA TYPE Representation of the objects of the data type Example: char char blah=‘A’; (‘A’: ASCII code is 65(dec), or 0x41 (hex)) Q: The maximum number which can be represented with a char variable? A: 255. Homework: How about char, int, long, float? 01000001 1 byte of memory: DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 25

26. DATA TYPE Q: Do we need to know about the representation of a data type? A: It depends. We can usually write algorithms which are more efficient if we make use of the knowledge about the representation. However!!! If the representation of the data type is changed, the program needs to be verified, revised, or completely re-written. 囧 Porting to a different platform (x86, ARM, embedded system, …) Changing the specification of a program or a library (ex. 16-bit int  32-bit long) DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 26

27. ABSTRACT DATA TYPE An “abstract data type” (ADT) is a data type that is organized in such a way that the specification of the objects and the specification of the operations on the objects is separated from the representation of the objects and the implementation of the operations. Representation and Implementation Specification (Interface) User DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 27

28. ABSTRACT DATA TYPE Specifications: Name of the function and the description of what the function does The type of the argument(s) The type of the result(s) (return value) Function categories: Creator/constructor Transformers Observer/reporter Homework: Read Example 1.5 on p. 20 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 28

29. PERFORMANCE ANALYSIS DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 29

30. HOW DO YOU MAKE EVALUATIVE JUDGMENT ABOUT PROGRAM? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 30 1.Does the program meet the original specifications of the task? 2.Does it work correctly? 3.Does the program contain documentation that shows how to use it and how it works? 4.Does the program effectively use functions to create logical units? 5.Is the program’s code readable?

31. HOW DO YOU MAKE EVALUATIVE JUDGMENT ABOUT PROGRAM? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 31 6.Does the program efficiently use primary and secondary storage? Primary storage: memory? Secondary storage: Hard drive, flash disk, etc. 7.Is the program’s running time acceptable for the task? Example: Network intrusion detection system (1) 99.8% detection rate, 50 minutes to finish analysis of a minute of traffic (2) 85% detection rate, 20 seconds to finish analysis of a minute of traffic

32. HOW DO YOU MAKE EVALUATIVE JUDGMENT ABOUT PROGRAM? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 32 6.Does the program efficiently use primary and secondary storage? 7.Is the program’s running time acceptable for the task?

33. SPACE & TIME COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 33 Space complexity of a program: The amount of memory that it needs to run to completion. Time complexity of a program: The amount of computer time that it needs to run to completion.

34. SPACE COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 34

35. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 35

36. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 36

37. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 37

38. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 38 Is it good to use? (recursion) TypeNameNumber of bytes parameters: array pointerlist[]4 parameter: integern4 return address: (used internally) 4 TOTAL per recursive call12 Probably not for this problem.

39. TIME COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 39

40. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 40 Method 1: Count all instances of all operations in the program. AddSubtractLoadStore ADD(n)SUB(n)LDA(n)STA(n) Is it good to use? (method 1)

41. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 41 Method 2: Separate the program into program steps whose execution time is independent of instance characteristics Count the number of steps Different program steps have different amounts of computing a=2; a=2*b+3*c/d-e+f/g/a/b/c; Count the number of steps needs to solve a particular instance

42. EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 42 float sum(float list[], int n) { float tempsum = 0; count++; //assignment int i; for (i=0;i<n;i++) { count++; // for loop tempsum+=list[i]; count++; //assignment } count++; //last iteration of for count++; return tempsum; }

43. EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 43 float sum(float list[], int n) { float tempsum = 0; int i; for (i=0;i<n;i++) { count+=2; } count+=3; return tempsum; } count = 2n+3 (steps)

44. EXAMPLE 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 44 float rsum(float list[], int n) { count++; // if if (n) { count++; //return return rsum(list,n-1)+list[n-1]; } count++; //return return list[0]; } count = 2n+2 (steps) 2n+2 < 2n+3. Does this mean rsum is faster than sum ? No!

45. EXAMPLE 3 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 45 void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int c[][MAX_SIZE], int rows, int cols) { int i,j; for(i=0; i<rows; ++i) for(j=0;j<cols;++j) c[i][j]=a[i][j]+b[i][j]; }

46. EXAMPLE 3 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 46 void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int c[][MAX_SIZE], int rows, int cols) { int i,j; for(i=0; i<rows; ++i) { count++; for(j=0;j<cols;++j) { count++; c[i][j]=a[i][j]+b[i][j]; count++; } count++; } count++; } count=((2*cols)+2)*rows+1 =2cols*rows+2*rows+1

47. EXAMPLE 1 REVISITED (TABULAR METHOD) DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 47 Statements/eFrequencyTotal steps float sum(float list[], int n) { 000 float tempsum=0; 111 int i; 000 for (i=0;i<n;i++) 0n+1 tempsum+=list[i]; 1nn return tempsum; } 111 Please practice using the method for the other two examples.

48. SUMMARY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 48 Instance characteristics? The number of inputs The number of outputs The magnitude of the inputs We then calculate the number of steps which is independent of the characteristics we selected.

49. DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 49 Sometimes, the time complexity does not depend on only these simple numbers. For example, binsearch. In that case, we are interested in the average case, the worst case, and the best case.

50. ASYMPTOTIC NOTATION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 50

51. MOTIVATION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 51 Compare the time complexities of two programs that compute the same function. Predict the growth in run time as the instance characteristics change. “Steps” are not exact. “3n+3” faster than “3n+5” ? Usually “3n+3”, “7n+2”, or “2n+15” all runs for about the same time. Let’s go for a less accurate way to describe the run time…

52. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 52 Break even point No.

53. ASYMPTOTIC NOTATION – BIG OH DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 53

54. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 54

55. THE WORLD OF BIG OH DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 55 Faster Slower

56. BIG OH IS AN UPPER BOUND DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 56

57. A USEFUL THEOREM DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 57

58. ASYMPTOTIC NOTATION – OMEGA DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 58

59. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 59

60. DISCUSSION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 60 Homework 1

61. ASYMPTOTIC NOTATION – THETA DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 61 Theta Platform GMC Terrain

62. EXAMPLE DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 62

63. DISCUSSION DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 63 Homework 1

64. SO, WHAT NOW? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 64 Asymptotic complexity can be determined quite easily Work on each statement first Then we add them up No need for the silly step counts anymore

65. EXAMPLE 1 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 65 StatementAsymptotic complexity void add(int a[][MAX_SIZE]…) {0 int i,j;0 for(i=0;i<rows;i++) for(j-0;j<cols;j++) c[i][j]=a[i][j]+b[i][j]; }0 Total

66. EXAMPLE 2 DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 66 int binsearch(int list[], int searchnum, int left, int right) { int middle; while(left<=right) { middle=(left+right)/2; switch(COMPARE(list[middle], searchnum)) { case -1: left=middle+1; break; case 0: return middle; case 1: right=middle-1; } return -1; }

67. 1344 6 71113 1819 0123456789 searchnum=13; left right middl e middle=(left+right)/2; left=middle+1; DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 67 1011 Half the searching window every iteration.

68. MORE EXAMPLES DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 68 Please take a look at example 1.20 and 1.21 (p. 38-40)

69. PRACTICAL COMPLEXITY DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 69

70. 兜基 ?? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 70

71. WHICH ONE IS BETTER? DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 71

72. ON A 1 BILLION-STEPS-PER-SEC COMPUTER DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 72 n

73. 下課 ! DATA STRUCTURE AND ALGORITHM I HSIN-MU TSAI, FALL 2010 73 等等 ~~ Homework 1 is out today, the due date is two weeks from yesterday (5pm). Download the question sets from the course website E-mail your question to dsa1@csie.ntu.edu.twdsa1@csie.ntu.edu.tw Or, come to our office hours. Have a nice weekend!