Design and Analysis of Algorithms Text Book: Horowitz, S. Sahni, Fundamentals of Computer Algorithms Reference Book: Introduction to algorithms  By Thomas H. Cormen

Notion: Algorithms An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. “computer” problem algorithm input output Algorithm is thus a sequence of computational steps that transform the input into the output.

More precisely, an algorithm is a method or process to solve a problem satisfying the following properties: Finiteness terminates after a finite number of steps Definiteness Each step must be rigorously and unambiguously specified. Input Valid inputs must be clearly specified. Output can be proved to produce the correct output given a valid can be proved to produce the correct output given a valid input. Effectiveness Steps must be sufficiently simple and basic. Can be carried out with pen and paper These are also called characteristics of an algorithm

Examples Is the following a legitimate algorithm? i 1 While (i <= 10) do a  i + 1 Print the value of a End of loop Stop

Four key terms: Natural Language Algorithm Program : A program is the expression of an algorithm in a programming language Psuedocode-mix of algorithm and some programming language. Key Points: Each step of an algorithm must be unambiguous. The same algorithm can be represented in several different ways. There might exists more than one algorithm for a certain problem. Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds.

Two main issues related to algorithms How to design algorithms How to analyze algorithm efficiency

Analysis of Algorithms How good is the algorithm? (Determined by the complexity) time efficiency space efficiency Does there exist a better algorithm? lower bounds optimality

Importance of Analyze Algorithm Need to recognize limitations of various algorithms for solving a problem Need to understand relationship between problem size and running time When is a running program not good enough? Need to learn how to analyze an algorithm's running time without coding it Need to learn techniques for writing more efficient code

What do we analyze about them? Correctness Does the input/output relation match algorithm requirement? Amount of work done (aka complexity) Basic operations to do task Amount of space used Memory used Simplicity, clarity Verification and implementation. Optimality Is it impossible to do better?

Models of Computation Reference: The design and analysis of computer algorithms by Aho Ullman

Three models of computation Random Access Machines RASP Machines(Stored program model) Turing Machines

Random Access Machine (RAM)

Each register holds an integer Program can’t modify itself RAM Assumptions Each register holds an integer Program can’t modify itself Memory instructions involve simple arithmetic Addition, subtraction Multiplication, division and control states (got, if-then, etc.)

RASP Machine stored program model Same as RAM but allow program to change itself as it is now stored in the memory Same power as RAM Example Von Neumann architecture Let RAM use memory registers to store modifiable program of RASP

Turing Machine A Turing machine includes A (conceptual) tape that extends infinitely in both directions Holds the input to the Turing machine Serves as memory Is divided into cells A unit that reads one cell of the tape at a time and writes a symbol in that cell It is controlled by finite automaton that has finite number of states.

Example of a finite automaton containing two states is a light switch. In this example, the two states are on and off. If the state of the lightswitch is on, pushing the switch is the input that will cause a transition to the off state. Each cell contains one symbol Symbols must come from a finite set of symbols called the alphabet Alphabet for a given Turing machine Contains a special symbol b (for “blank”) Usually contains the symbols 0 and 1 Sometimes contains additional symbols

Each operation involves Writing a symbol in the cell (replacing the symbol already there) Going into a new state (could be same state) Moving one cell left or right

Each instruction says something like if (you are in state i) and (you are reading symbol j) then write symbol k onto the tape go into state s move in direction d