1. Algorithm Analysis Lectures 3 & 4
Resources
Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2
Introduction to Algorithms (Cormen, Leiserson, & Rivest): Chap.1
Algorithms Theory & Practice (Brassard & Bratley): Chap. 1

2. Algorithms
An algorithm is a well-defined computational procedure that takes some value or a set of values, as input and produces some value, or a set of values as output.
Or, an algorithm is a well-specified set of instructions to be solve a problem.

3. Efficiency of Algorithms
Empirical
Programming competing algorithms and trying them on different instances
Theoretical
Determining mathematically the quantity of resources (execution time, memory space, etc) needed by each algorithm

4. Analyzing Algorithms
Predicting the resources that the algorithm requires:
Computational running time
Memory usage
Communication bandwidth
The running time of an algorithm
Number of primitive operations on a particular input size
Depends on
Input size (e.g. 60 elements vs )
The input itself ( partially sorted input for a sorting algorithm)

5. Order of Growth The order (rate) of growth of a running time
Ignore machine dependant constants
Look at growth of T(n) as n
 notation
Drop low-order terms
Ignore leading constants
E.g.
3n n2 – 2n +5 =  (n3)

6. Mathematical Background

7. Mathematical Background
Definitions:
T(N) = O(f(N)) iff  c and n0 
T(N)  c.f(N) when N  n0
T(N) = (g(N)) iff  c and n0 
T(N)  c.g(N) when N  n0
T(N) = (h(N)) iff T(N) = O(h(N)) and
T(N) = (h(N))

8. Mathematical Background
Definitions:
T(N) = o(f(N)) iff  c and n0 
T(N)  c.f(N) when N  n0
T(N) = (g(N)) iff  c and n0 
T(N)  c.g(N) when N  n0

9. Mathematical Background
Rules:
If T1(N) = O(f(N)) and T2(N) = O(g(N)) then
a) T1(N) + T2(N) = max( O(f(N)),O(g(N))
b) T1(N) * T2(N) = O(f(N) * g(N))
If T(N) is a polynomial of degree k, then T(N) = (Nk)
Logk N = O(N) for any constant k.

10. More … 3n3 + 90n2 – 2n +5 = O(n3 ) 2n2 + 3n +1000000 = (n2)
2n = o(n2) ( set membership)
3n2 = O(n2) tighter (n2)
n log n = O(n2)
True or false:
n2 = O(n3 )
n3 = O(n2)
2n+1= O(2n)
(n+1)! = O(n!)

11. Ranking by Order of Growth
1 n n log n n2 nk
(3/2)n 2n (n)! (n+1)!

12. Running time calculations
Rule 1 – For Loops
The running time of a for loop is at most the running time of the statement inside the for loop (including tests) times the number of iterations
Rule 2 – Nested Loops
Analyze these inside out. The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops

13. Running time calculations: Examples
sum = 0;
for (i=1; i <=n; i++)
sum += n;
Example 2:
for (j=1; j<=n; j++)
for (i=1; i<=j; i++)
sum++;
for (k=0; k<n; k++)
A[k] = k;

14. How to rank?