Design & Analysis of Algorithms Lecture 1 Introduction

Sequence of steps that can be taken to solve a problem An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time. An Algorithm is well defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output. More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output. Algorithm Inputoutput What is Algorithm? Mashhood's Web Family2

It must be correct. It must be composed of a series of concrete steps. The execution sequence of instructions should not be ambiguous. It must have finite number of instructions and steps. It must terminate. 3 Mashhood's Web Family3

Syntax & Semantics An algo. is “correct” if its: –Semantics are correct –Syntax is correct Semantics: The concept embedded in an algorithm (the soul!) Syntax: The actual representation of an algorithm (the body!) WARNINGS: 1. An algo. can be syntactically correct, yet semantically incorrect – very dangerous situation! 2. Syntactic correctness is easier to check as compared with semantic Mashhood's Web Family4

Solving Problems (1) When faced with a problem: 1.We first clearly define the problem 2.Think of possible solutions 3.Select the one that we think is the best under the prevailing circumstances 4.And then apply that solution 5.If the solution woks as desired, fine; else we go back to step 2 Mashhood's Web Family5

Solving Problems(2) It is quite common to first solve a problem for a particular case Then for another And, possibly another And watch for patterns and trends that emerge And to use the knowledge form those patterns and trends in coming up with a general solution And this general solution is called ……………. Mashhood's Web Family6

Early History: Search for a Generic Algorithm The study of algorithms began with mathematicians and was a significant area of work in the early years The goal of those early studies was to find a single, general algorithm that could solve all problems of a single type Mashhood's Web Family7

Origin of the Term “Algorithm” The name derives from the title of a Latin book: Algoritmi de numero Indorum That book was a translation of an Arabic book: Al-Khwarizmi Concerning the Hindu Art of Reckoning That book was written by the famous 9-th century Muslim mathematician, Muhammad ibn Musa al-Khwarizmi Mashhood's Web Family8

Al-Khwarzmi Al-Khwarizmi lived in Baghdad, where he worked at the Dar al-Hikma Dar al-Hikma acquired and translated books on science and philosophy, particularly those in Greek, as well as publishing original research The word Algebra has its origins in the title of another Latin book which was a translation of yet another book written by Al-Khwarzmi: Kitab al-Mukhtasar fi Hisab al-Jabr wa'l-Muqabala Mashhood's Web Family9

Al-Khwarizmi’s Golden Principle All complex problems can be and must be solved using the following simple steps: 1.Break down the problem into small, simple sub-problems 2.Arrange the sub-problems in such an order that each of them can be solved without effecting any other 3.Solve them separately, in the correct order 4.Combine the solutions of the sub-problems to form the solution of the original problem Mashhood's Web Family10

11 Problem High Level Language Program High Level Language Program Algorithm Algorithm : A sequence of instructions describing how to do a task Mashhood's Web Family11

Algorithm It is a method followed to solve a problem. A mapping of input to output. As mentioned on slide no. 2. A problem can have many algorithms Computer program It is a concrete representation of an algorithm in some programming language A set of instructions which the computer will follow to solve a problem 12 Mashhood's Web Family12

Why Algorithms are Useful? Once we find an algorithm for solving a problem, we do not need to re-discover it the next time we are faced with that problem Once an algorithm is known, the task of solving the problem reduces to following (almost blindly and without thinking) the instructions precisely All the knowledge required for solving the problem is present in the algorithm Mashhood's Web Family13

Why Write an Algorithm Down? For your own use in the future, so that you don’t have spend the time for rethinking it Written form is easier to modify and improve Makes it easy when explaining the process to others Mashhood's Web Family14

Problem and Instances An input sequence is called an instance of a Problem A problem has many particular instances An algorithm must work correctly on all instances of the problem it claims to solve Many interesting problems have infinitely many instances –Since computers are finite, we usually need to limit the number and/or size of possible instances in this case 15Mashhood's Web Family

Most basic and popular algorithms are –Sorting algorithms –Searching algorithms Which algorithm is best? Mainly, it depends upon various factors, for example in case of sorting –The number of items to be sorted –The extent to which the items are already sorted –Possible restrictions on the item values –The kind of storage device to be used etc. Popular Algorithms, Factors of Dependence Mashhood's Web Family16

Problem The statement of the problem specifies, in general terms, the desired input/output relationship. Algorithm The algorithm describes a specific computational procedure for achieving input/output relationship. Example One might need to sort a sequence of numbers into non-decreasing order. Algorithms Various algorithms e.g. merge sort, quick sort, heap sorts etc. One Problem, Many Algorithms Mashhood's Web Family17

Brute Force –Straightforward, naive approach –Mostly expensive Divide-and-Conquer –Divide into smaller sub-problems Iterative Improvement –Improve one change at a time Decrease-and-Conquer –Decrease instance size Transform-and-Conquer –Modify problem first and then solve it Important Designing Techniques Mashhood's Web Family18

Greedy Approach –Locally optimal decisions, can not change once made. –Efficient –Easy to implement –The solution is expected to be optimal –Every problem may not have greedy solution Dynamic programming –Decompose into sub-problems like divide and conquer –Sub-problems are dependant –Record results of smaller sub-problems –Re-use it for further occurrence –Mostly reduces complexity exponential to polynomial There is a Tradeoffs b/w Space and Time –Use more space now to save time later Important Designing Techniques Mashhood's Web Family19

Analysis –How does system work? –Breaking a system down to known components –How components (processes) relate to each other –Breaking a process down to known functions Synthesis –Building tools –Building functions with supporting tools –Composing functions to form a process –How components should be put together? –Final solution Problem Solving Phases Mashhood's Web Family20

Problem Strategy Algorithm –Input –Output –Steps Analysis –Correctness –Time & Space –Optimality Implementation Verification Problem Solving Process Mashhood's Web Family21

Our today’s lecture was on basic introduction of the Algorithms In next lecture we will talk about the Analysis of Algorithms INSHALLAH Mashhood's Web Family22