1. College of Information Technology & Design
Discrete Mathematics
Complexity of Algorithms
University of Jazeera
College of Information Technology & Design
Khulood Ghazal

2. Complexity of Algorithms
The Time Complexity of an algorithm can be expressed in terms of the number of basic operations used by the algorithm when the input has a particular size.
Examples of basic operations are :
Additions
Multiplications
Comparison of two numbers
The space complexity of an algorithm is expressed in terms of the memory required by the algorithm for an input of a particular size.
Type of complexity analysis:
Worst case: The largest number of operations needed to solve the problem of specified size.
Average case: the average number of operations used to solve the problem over all inputs of a given size
Use the Big-O notation

3. Example:
Describe the time complexity for algorithm of finding the maximum element in a set.
The basic operation used is the comparison .The number of comparison will be used as the measure of the time complexity of the algorithm.
From the second term (a2) we have 2 comparison for each term of the list:
1.To determine that the end of list has not been reached .
2.To determine whether to update the max.
we have one more comparison is used to exit the loop when i=n+1.
So total comparisons are :
2(n-1) +1=2n-1
procedure max ( a1,a2,…,an : integers )
max := a1
for i := 2 to n
if ai > max then max := ai
return max { max is the largest element }
So this algorithm has time complexity O(n).

4. Describe the time complexity of the linear search algorithm.
Example 2:
Describe the time complexity of the linear search algorithm.
The basic operation used is the comparison .The number of comparison will be used as the measure of the time complexity of the algorithm.
procedure linear search (x: integer, a1, a2, …, an: distinct integers)
i := while (i  n  x  ai ) i := i + 1
End if i  n then location := i else location := 0
return location {index or 0 if not found}
At each step of the loop in the algorithm , 2 comparisons are performed :
See whether the end of the list has been reached.
Compare the element x with a term of the list.
One more comparison is made outside the loop .
So time complexity is : 2n+1=O( n).
Worst case analysis:
When the element is not in the list ,2n comparisons are used to determine
that x is not in the list , an additional comparison is used to exit the loop, and
one comparison is made outside the loop so a total of 2n+1+1 comparison are
used .
So worst case is O(n) .

5. Describe the average-case of the linear search assuming that the element x is in the list.
If x is the first term of the list ,three comparison are needed ,one to determine if the end of the list has been reached , one to compare x and the first term, and one outside the loop.
If x is the second term of the list ,two comparison are needed , so that a total of five comparison are used .
If x is the third term of the list ,two comparison are needed , so that a total of seven comparison are used .
So if x is the ith term of the list ,two comparison will be used at each of the i steps and one outside the loop so the total of 2i+1 comparison are needed.
The average number of comparisons used equal:
=(3+5+7+…………+(2n+1)/n )
which is O(n)

6. Describe the time complexity of the binary search algorithm .
Example:
Describe the time complexity of the binary search algorithm .
Assume n=2k (where k is a positive integer ),k=log2 n.
Two comparison are used at the first stage of the algorithm when the list has 2k elements .
Two more when the search has been reduced to a list with 2k-1 elements .
Two more when the search has been reduced to a list with 2k-2 elements .
Two more when the search has been reduced to a list with 2k-3 elements
……………………..and so on.
procedure binary search(x:integer, a1, a2, …, an:
distinct integers)
i := 1 {left endpoint of search interval} j := n {right endpoint of search interval} while i < j begin {while interval has >1 item} m := (i + j ) /2 {midpoint} if x > am then i := m+1 else j := m end if x = ai then location := i else location := 0 return location
Finally ,when one term is left in the list one comparison tell us that there are no additional term left ,and one more comparison is used to determine if this term is x.
So 2*k+2=2*log2n+2 comparison are required.
So the worst - case of the binary search is O(log2n) . .

7. Is Binary Search more efficient than linear search?
Number of iterations:
For a list of n elements, Binary Search can execute at most log2 n times!!
Linear Search, on the other hand, can execute up to n times !!
Number of computations per iteration:
Binary search does more computations than Linear Search per iteration.
Overall:
If the number of components is small (say, less than 20), then Linear Search is faster.
If the number of components is large, then Binary Search is faster.

8. if (i<j) for ( i=0; i<N; i++ ) X = X+i; O(N) else X=0;
Examples: Describe the time complexity of the following algorithm:
if (i<j)
for ( i=0; i<N; i++ )
X = X+i;
else
X=0;
Max ( O(N), O(1) ) = O (N)
O(N)
O(1)

9. Examples: Describe the time complexity of the algorithm:
• Multiply an n x n matrix A by a scalar c to produce the matrix B:
procedure (n, c, A, B)
for i from 1 to n do
for j from 1 to n do
B(i, j) = c*A(i, j)
end do
Analysis (worst case):
Count the number of multiplications.
n2 elements requires n2 multiplications.
Time complexity is = O(n2)