1. Algorithm Analysis and DesignBy
Muhammad Munawar Ahmed

2. Objective of This Course
Major objective of this course is:
Design and analysis of modern algorithms
Different variants
Accuracy
Efficiency
Comparing efficiencies
Motivation thinking new algorithms
Advanced designing techniques
Real world problems will be taken as examples
To create feelings about usefulness of this course

3. Introduction (What, Why and Where uses Algorithms)

4. Today Discussion In this lecture we will cover the following
What is Algorithm?
Designing Techniques
Model of Computation
Algorithms as a technology
Algorithms and other technologies
Importance of algorithms
Difference in Users and Developers
Kinds of problems solved by algorithms
Conclusion

5. What is Algorithm? Algorithm Input output
A computer algorithm is a detailed step-by-step method for solving a problem by using a computer.
An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time which can be implemented on a computer.
An Algorithm is well defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output.
More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output.
Algorithm
Input
output
Algorithms Analysis and Design

6. What is Algorithm? A Problem must be computational problems.
Examples of computational problems
Finding max value from a set of differnt values.
Sorting values in ascending or descending order.
Product of list of values.
Examples of non-computational problems
What is cure of malaria?
How do I convince myslef to select a course?

7. implementation depends on the language.
BASIC ALGORITHMS
Several algorithms are used in computer science so prevalently that they are considered “basic”. We discuss the most common here. This discussion is very general:
implementation depends on the language.

8. Summation
One commonly used algorithm in computer science is summation. We can add two or three integers very easily, but how can we add many integers? The solution is simple: we use the add operator in a loop (Figure 8.9).
A summation algorithm has three logical parts:
1. Initialization of the sum at the beginning.
2. The loop, which in each iteration adds a new integer to the sum.
3. Return of the result after exiting from the loop.

9. Figure 8.9 Summation algorithm

10. Product
Another common algorithm is finding the product of a list of integers. The solution is simple: use the multiplication operator in a loop .
A product algorithm has three logical parts:
1. Initialization of the product at the beginning.
2. The loop, which in each iteration multiplies a new integer with the product.
3. Return of the result after exiting from the loop.

11. Product algorithm

12. Smallest and largest
We discussed the algorithm for finding the largest among a list of integers at the beginning of this chapter. The idea was to write a decision construct to find the larger of two integers. If we put this construct in a loop, we can find the largest of a list of integers.
Finding the smallest integer among a list of integers is similar, with two minor differences. First, we use a decision construct to find the smaller of two integers. Second, we initialize with a very large integer instead of a very small one.

13. Sorting
One of the most common applications in computer science is sorting, which is the process by which data is arranged according to its values. People are surrounded by data. If the data was not ordered, it would take hours and hours to find a single piece of information. Imagine the difficulty of finding someone’s telephone number in a telephone book that is not ordered.
In this section, we introduce three sorting algorithms: selection sort, bubble sort and insertion sort. These three sorting algorithms are the foundation of faster sorting algorithms used in computer science today.

14. Selection sorts
In a selection sort, the list to be sorted is divided into two sublists—sorted and unsorted—which are separated by an imaginary wall. We find the smallest element from the unsorted sublist and swap it with the element at the beginning of the unsorted sublist. After each selection and swap, the imaginary wall between the two sublists moves one element ahead.
Figure Selection sort

15. Figure 8.12 Example of selection sort

16. Figure 8.13 Selection sort algorithm

17. Bubble sorts
In the bubble sort method, the list to be sorted is also divided into two sublists—sorted and unsorted. The smallest element is bubbled up from the unsorted sublist and moved to the sorted sublist. After the smallest element has been moved to the sorted list, the wall moves one element ahead.
Figure Bubble sort

18. Figure 8.15 Example of bubble sort

19. Insertion sorts
The insertion sort algorithm is one of the most common sorting techniques, and it is often used by card players. Each card a player picks up is inserted into the proper place in their hand of cards to maintain a particular sequence.
Figure Insertion sort

20. Figure 8.17 Example of insertion sort

21. Searching
Another common algorithm in computer science is searching, which is the process of finding the location of a target among a list of objects. In the case of a list, searching means that given a value, we want to find the location of the first element in the list that contains that value. There are two basic searches for lists: sequential search and binary search. Sequential search can be used to locate an item in any list, whereas binary search requires the list first to be sorted.

22. Sequential search
Sequential search is used if the list to be searched is not ordered. Generally, we use this technique only for small lists, or lists that are not searched often. In other cases, the best approach is to first sort the list and then search it using the binary search discussed later.
In a sequential search, we start searching for the target from the beginning of the list. We continue until we either find the target or reach the end of the list.

23. Figure 8.18 An example of a sequential search

24. Binary search
The sequential search algorithm is very slow. If we have a list of a million elements, we must do a million comparisons in the worst case. If the list is not sorted, this is the only solution. If the list is sorted, however, we can use a more efficient algorithm called binary search. Generally speaking, programmers use a binary search when a list is large.
A binary search starts by testing the data in the element at the middle of the list. This determines whether the target is in the first half or the second half of the list. If it is in the first half, there is no need to further check the second half. If it is in the second half, there is no need to further check the first half. In other words, we eliminate half the list from further consideration.

25. Figure 8.19 Example of a binary search

26. SUBALGORITHMS
In Some cases an algorithm be broken into small units called subalgorithms. Each subalgorithm is in turn divided into smaller subalgorithms. A good example is the algorithm for the selection sort.

27. Figure 8.20 Concept of a subalgorithm

28. RECURSION
In general, there are two approaches to writing algorithms for solving a problem. One uses iteration, the other uses recursion. Recursion is a process in which an algorithm calls itself.

29. Iterative definition
To study a simple example, consider the calculation of a factorial. The factorial of a integer is the product of the integral values from 1 to the integer. The definition is iterative . An algorithm is iterative whenever the definition does not involve the algorithm itself.
Figure Iterative definition of factorial

30. Recursive definition
An algorithm is defined recursively whenever the algorithm appears within the definition itself. For example, the factorial function can be defined recursively.
Figure Recursive definition of factorial

31. Figure 8.23 Tracing the recursive solution to the factorial problem

32. Iterative solution
This solution usually involves a loop.

33. Recursive solution
The recursive solution does not need a loop, as the recursion concept itself involves repetition.

34. What is Algorithm Analysis?
Algorithm: A clearly specified finite set of instructions a computer follows to solve a problem.
Algorithm analysis:
Predicting the resources that an algorithm requires. i.e computational time and memory.
a process of determining the amount of time, resource, etc. required when executing an algorithm.

35. What is Algorithm Analysis?
When there are multiple, alternative algorithms, we analyse them and pick the most efficeint one. e. g different sorting algorithms.
Problem: Exact No of resources depends upon program, programming language, hardware and any kind of other job that processor is executing.
Solution: Define a generic machine and imagine running of algorithm on it.

36. Problem Solving Process
Strategy
Algorithm
Input
Output
Steps
Analysis
Correctness
Time & Space
Optimality
Implementation
Verification

37. Model of Computation (Assumptions)
Design assumption
Level of abstraction which meets our requirements
Neither more nor less e.g. [0, 1] infinite continuous interval
Analysis independent of the variations in
Machine
Operating system
Programming languages
Compiler etc.
Low-level details will not be considered
Our model will be an abstraction of a standard generic single-processor machine, called a random access machine or RAM.

38. Model of Computation (Assumptions)
A RAM is assumed to be an idealized machine
Infinitely large random-access memory
Instructions execute sequentially
Every instruction is in fact a basic operation on two values in the machines memory which takes unit time.
These might be characters or integers.
Example of basic operations include
Assigning a value to a variable
Arithmetic operation (+, - , × , /) on integers
Performing any comparison e.g. a < b
Boolean operations
Accessing an element of an array.

39. Model of Computation (Assumptions)
In theoretical analysis, computational complexity
Estimated in asymptotic sense, (in computer science in the analysis of algorithms, considering the performance of algorithms when applied to very large input datasets).i.e. Estimating for large inputs
Big O, Omega, Theta etc. notations are used to compute the complexity
Asymptotic notations are used because different implementations of algorithm may differ in efficiency
Efficiencies of two given algorithm are related
By a constant multiplicative factor
Called hidden constant.
Algorithms Analysis and Design

40. Drawbacks in Model of Computation
First poor assumption
We assumed that each basic operation takes constant time, i.e. model allows
Adding
Multiplying
Comparing etc.
two numbers of any length in constant time
Addition of two numbers takes a unit time!
Not good because numbers may be arbitrarily
Addition and multiplication both take unit time!
Again very bad assumption

41. Model of Computation not so Bad
Finally what about Our Model?
But with all these weaknesses, our model is not so bad because we have to give the
Comparison not the absolute analysis of any algorithm.
We have to deal with large inputs not with the small size
Model seems to work well describing computational power of modern nonparallel machines
Can we do Exact Measure of Efficiency ?
Exact, not asymptotic, measure of efficiency can be sometimes computed but it usually requires certain assumptions concerning implementation
Algorithms Analysis and Design

42. A Simple Example // Input: int A[N], array of N integers
// Output: Sum of all numbers in array A
int Sum(int A[], int N) {
int s=0;
for (int i=0; i< N; i++)
s = s + A[i];
return s; }
How should we analyse this?
Applied Algo Fall-03

43. A Simple Example Analysis of Sum
1.) Describe the size of the input in terms of one ore more
parameters:
- Input to Sum is an array of N ints, so size is N.
2.) Then, count how many steps are used for an input of that size:
- A step is an elementary operation such as
+, <, =, A[i]
Applied Algo Fall-03

44. A Simple Example Analysis of Sum (2) 1,2,8: Once
// Input: int A[N], array of N integers
// Output: Sum of all numbers in array A
int Sum(int A[], int N {
int s=0;
for (int i=0; i< N; i++)
s = s + A[i];
return s; } 1 2 3 4 5
1,2,8: Once
3,4,5,6,7: Once per each iteration
of for loop, N iteration
Total: 5N + 3
The complexity function of the
algorithm is : f(N) = 5N +3 6 7 8
Applied Algo Fall-03

45. Analysis: A Simple Example How 5N+3 Grows
Estimated running time for different values of N:
N = 10 => 53 steps
N = 100 => 503 steps
N = 1,000 => 5003 steps
N = 1,000,000 => 5,000,003 steps
As N grows, the number of steps grow in linear proportion to
N for this Sum function.
Applied Algo Fall-03

46. Summary : Computational Model
Analysis will be performed with respect to this computational model for comparison of algorithms
We will give asymptotic analysis not detailed comparison i.e. for large inputs
We will use generic uniprocessor random-access machine (RAM) in analysis
All memory equally expensive to access
No concurrent operations
All reasonable instructions take unit time, except, of course, function calls