1. Algorithm Analysis

2. Question
Suppose you have 2 programs that will sort a list of student records and allow you to search for student information. Question: How do you judge which is a better program? Answer:
By running programs or
Algorithm Analysis

3. Algorithm Analysis
Is a methodology for estimating the resource (time and space) consumption of an algorithm.
Allows us to compare the relative costs of two or more algorithms for solving the same problem.

4. How to Measure “Betterness”
Critical resources in computer
Time
Memory space
For most algorithms, running time depends on size n of data.
Notation: T(n)
Time T is a function of data size n.

5. Two approaches Two approaches to obtaining running time:
Measuring under standard benchmark conditions (running programs)
Estimating the algorithms performance (analyzing algorithms)

6. Estimation Assumptions
Estimation of running time is based on:
Size of the input
Number of basic operations
The time to complete a basic operation does not depend on the value of its operands.

7. Example: largest() // Return position of largest value // in array
int largest(int array[], int n) {
int posBig = 0; // 1
for (int i = 1; i < n; i++) if (array[posBig] < array[i]){ // 2
posBig = i; // 3 }
return posBig;
As n grows, how does T(n) grow?
Cost: T(n) = c1n + c2 steps

8. Example: largest() Step 1: T(n) = c1
Step 2: It takes a fixed amount of time to do one comparison T(n) = c2n
Step 3: T(n) = c3n, at most
Total time: T(n) = c1 + c2n+ c3n ≈ c1 + (c2 + c3)n ≈ cn

9. Example: Assignment int a = array[0]; The running time is T(n) = c
This is called constant running time.

10. Example: total() sum = 0; for (i = 1; i <= n; i++)
for (j = 1; j < n; j++)
sum++; }
What is the running time for this code?
Analysis:
The basic operation is sum++
The cost of a sum operation can be bundled into some constant time c.
Inner loop takes c1n; outer loop takes c2n.
The total running time is T(n) = c1n * c2n = (c1 + c2)n = c3n2

11. Growth Rate of Algorithm —Big-O Natation
Growth Rate for an algorithm
is the rate at which the cost of the algorithm grows as the data size n grows.
Constant: T(n) = c  O(1)
Linear Growth: T(n) = n  O(n)
Quadratic Growth: T(n) = n2  O(n2)
Exponential Growth: T(n) = 2n  O(2n)
Assumptions
Growth rates are estimates.
They have meaning when n is large.

12. Growth Rate Graph

13. Growth Rate Graph c
Log n
T(n) n

14. Characteristics of Growth Rates
Constant— T(n) = c
Independent of n
Linear— T(n) = cn
Constant slope
Logarithmic— T(n) = c log n
Slope flattens out quickly
Quadratic— T(n) = cn2
Increasing slope
Cubic— T(n) = cn3
Exponential— T(n) = c2n

15. Growth Rate—Array Operations
bool search(int list[], int n, int key)
found = false Loop (for i = 0 to i < n) If (list[i] == key) Then found = true End If End loop
return found
End // search

16. Growth Rate—Array Operations
Linear Search
Insert at Back
Insert at Front
Remove from Back
Remove from Font

17. Linear Search bool search(int list[], int n, int key)
found = false // c Loop (for i = 0 to i < n) If (list[i] == key) Then // nc found = true (at most) End If End loop
return found // c3
End // search
T(n) = c1 + nc2 + c3 = c + nc2 ≈ nc2 ∴ O(n)

18. Insert at Back void insertAtBack(int list[], int &n, item)
If (!isFull) Then list[n] = item n++
End If
End // insertAtBack

19. Insert at Back void insertAtBack(int list[], int &n, item)
If (!isFull) Then list[n] = item // c1 n // c2
End If
End // insertAtBack
T(n) = c1 + c2 = c ∴ O(1)

20. Insert At Front void insertAtFront(int list[], int &n, int item)
If (!isFull) Then // shift to right Loop (for i = n downto 1) list[i] = list[i – 1] End loop // assign list[0] = item count++
End If
End // insertAtFront

21. Insert At Front—Growth Rate?
void insertAtFront(int list[], int &n, int item)
If (!isFull) Then Loop (for i = n downto 1) list[i] = list[i – 1] // c1n (at most) End loop list[0] = item // c2 count // c3
End If
End // insertAtFront
T(n) = c1n + c2 + c3 = nc1 + c ≈ nc1 ∴ O(n)

22. Remove From Back int removeFromBack(list[], int &n)
result = NIL If (!isEmpty) Then
result = list[n – 1] n = n End If
return result
End // removeFromBack

23. Remove From Front int removeFromFront(list[], int &n)
result = NIL If (!isEmpty) Then result = list[0] // shift left
Loop (for i = 0 to n – 2) list[i] = list[i + 1] End Loop
n = n End If
return result
End // removeFromFront

24. Growth Rate —Linked-List Operations
Linear Search
Insert at Back
Insert at Front
Remove from Back
Remove from Font

25. Linear Search
bool linearSearch(int key){ bool found = false; c1 if (head != NULL){ node* temp = head; c2 while (temp != NULL){ if (temp=>getData() != key) found = true;
else temp = temp->getNext(); c3n
} } return found;
} T(n) = c1 + c2 + c3n ≈ cn ∴ Growth Rate: O(n)

26. insertAtBack()
void insertAtBack(int item){ node *newP = new node(item, NULL); node *temp = head; c1
if (head == NULL) head = newP; else
while (temp->getNext() != NULL){ temp = temp->getNext(); c2n }
temp->setNext(newP); c3
count++; c4 } T(n) = c1 + c2n + c3 + c4 ≈ c + c2n ∴ G.R. = O(n)

27. insertAtFront()
void insertAtFront(int item){ node *newP = new node(item, NULL); c1 newP->setNext(head); c2 head = newP; Next(newP); c3
count++; c4 }
T(n) = c1 + c2 + c3 + c3 = c ∴ G.R. = O(1)

28. removeFromBack()
void removeFromBack(){ node *temp = head; c1 node *tail = NULL; c2 while (temp != NULL){ trail = temp; nc3
temp = temp->getNext(); nc4 }
count++; c5 }
T(n) = c1 + c2 + nc3 + nc4 + c5 = c6 + n(c3 + c4)
≈ cn
∴ G.R. = O(n)

29. removeFromFront()
void removeFromBack(){ node *newP = new node(item, NULL); c1 newP->setNext(head); c2 head = newP; Next(newP); c3
count++; c4 }
T(n) = c1 + c2 + c3 + c3 = c ∴ G.R. = O(1)

30. Best, Worst, Average Cases
For an algorithm with a given growth rate, we consider
Best case
Worst case
Average case
For example: Sequential search for K in an array of n integers
Best case: The first item of the array equals K
Worst case: The last position of the array equals K
Average case: Match at n/2
Best: Find at first position. Cost is 1 compare.
Worst: Find at last position. Cost is n compares.
Average: (n+1)/2 compares IF we assume the element with value K is equally likely to be in any position in the array.

31. Which Analysis to Use The Worst Case.
Useful in many real-time applications.
Advantage:
Predictability You know for certain that the algorithm must perform at least that well.
Disadvantage:
Might not be a representative measure of the behavior of the algorithm on inputs of size n.

32. Which Analysis to Use? The average case.
Often we prefer to know the average-case running time.
The average case reveals the typical behavior of the algorithm on inputs of size n.
Average case estimation is not always possible.
For the sequential search example, it assumes that the key value K is equally likely to appear in any position of the array. This assumption is not always correct.

33. Which Analysis to Use The Best Case
Normally, we are not interested in the best case, because:
It is too optimistic
Not a fair characterization of the algorithms’ running time
Useful in some rare cases, where the best case has high probability of occurring.

34. The moral of the story
If we know enough about the distribution of our input we prefer the average-case analysis.
If we do not know the distribution, then we must resort to worst-case analysis.
For real-time applications, the worst-case analysis is the preferred method.

35. Your Turn (Stack w/array)
Given Stack Class:
… private: elemType data[MAX]; int top; }

36. Stack (w/array) What is the growth rate (big O) of the push operation?
void push(elemType item){ if (!isFull()){ top++; data[top] = item; } }

37. Stack (w/array) What is the growth rate (big O) of the pop operation?
elemType pop (){ elemType result = NIL; if (!isEmpty()){ result = data[top]; top--; } }

38. Stack (w/array)
What is the growth rate (big O) of the clear operation?
void clear(){ top = -1; }

39. Stack (w/linked list) Given: Stack class with linked list
… private: Node *top; }

40. Stack (w/linked list)
What is the growth rate (big O) of the push operation?
void push(elemType item){ Node *temp = new Node(item, NULL); top->setNext(temp);
top = temp; }

41. Stack (w/linked list)
What is the growth rate (big O) of the pop operation?
elemType pop () { elemType result = NIL; Node *temp = top; if (!isEmpty()){ result = top-<getData();
top = top->getNext();
delete temp;
} return result; }

42. Stack (w/linked list)
What is the growth rate (big O) of the clear operation?
Void clear () { Node *temp = top; while (top != NULL) { top = top->getNext(); delete temp; temp = top; } }

43. Growth Rate of Binary Search (w/array)
Comparisons Number left n n(1/2) n(1/2)2
n(1/2)3
n(1/2)4
… …
k n(1/2)k-1
k n(1/2)k
Solution
n(1k) n
n(1/2)k = = ---- = (2k) (2k)
2k = n
log(2k) = log(n) k log(2) = log(n)
k = log(n)/log(2) k = c1 log(n)
T(n) = c1 c2 log(n)O(log n)

44. Analysis of Bubble Sort
Suppose: count = 10 maximum pass = 9 pass comparisons
void bubbleSort(int a[], int count) pass  count – last  pass – Loop (for i from 1 to pass) Loop (for j from 0 to last) If (a[j] > a[j + 1]) Then swap a[j] and a[j = 1] End If End Loop last  last – End Loop

45. Analysis of Bubble Sort
Total number of comparisons = ? = ? …………………… = 9x10
(n-1) + (n-2) + (n-3)… …+ (n-3) + (n-2) + (n-1) n n n … + n n n
= n(n-1)
Therefore, ? = n(n – 1)/ = cn2 – cn ≈ cn2
Growth Rate: O(n2)