Presentation is loading. Please wait.

Presentation is loading. Please wait.

CS4026 Formal Models of Computation Part III Computability & Complexity Part III-A – Computability Theory.

Similar presentations

Presentation on theme: "CS4026 Formal Models of Computation Part III Computability & Complexity Part III-A – Computability Theory."— Presentation transcript:

1 CS4026 Formal Models of Computation Part III Computability & Complexity Part III-A – Computability Theory

2 formal models of computation 2 Formal models of computation Before studying Haskell, we briefly examined the lambda calculus, a formal model underlying Haskell We shall soon explore another model of computation (called the imperative model), which underlies languages such as FORTRAN, PASCAL, C, JAVA etc: –Turing Machines Lets step back a second: Why study formal models (instead of only specific implementations)?

3 formal models of computation 3 Why study formal models of comp.? We do this to know what it means for a problem to be solvable by means of computation –Which problems are computable in a given type of programming language? –Best addressed by looking at a small prog. language As it happens, the answer does not depend on the type of programming language (functional / logical / imperative) –The equivalence of various models of computation is known as Churchs Thesis This course: address computability by focussing on the imperative model (i.e., Turing Machines)

4 formal models of computation 4 The Church-Turing Thesis Overview (Chapter 3 of Sipsers Book) Turing Machines Variants of Turing Machines Original motivation – find answers to these questions: What is an algorithm? What basic operations should algorithms contain?

5 formal models of computation 5 Turing Machines Abstract but accurate model of computers Proposed by Alan Turing in 1936 –There werent computers back then! Turings motivation: find out whether there exist mathematical problems that cannot be solved algorithmically. (This is Hilberts Entscheidungsproblem, i.e., decision problem) Similar to a FSA but with an –Unlimited and unrestricted memory Able to do everything a real computer can do However: –There are problems even a Turing Machine (TM) cant solve! –Such problems are beyond the theoretical limits of computation

6 formal models of computation 6 Turing Machines: Features An tape is a countably infinite list of cells A tape head can –read and write symbols from/onto a cell on the tape –move forward and backward on the tape Schematically: Control Head abaa... Tape

7 formal models of computation 7 Turing Machines: Features (Contd) Initially the tape –Contains only the input string –Is blank everywhere else If the TM needs to store information –It can write it onto the tape To read the info it has written –TM can move the head back over it TMs have the following behaviours: –They compute then stop in a reject state –They compute then stop in an accept state –They loop forever Compare FSAs: These have no reject states, and no loops. (Why no loops?)

8 formal models of computation 8 Turing Machines vs. Finite Automata TMs can both write on the tape and read from it –FSAs can only read (or only write if they are generators) The read/write head can move to the left and right –FSAs cannot go back on the input The tape is infinite –In FSAs the input and output is always finite The accept/reject states take immediate effect –There is no need to consume all the input TMs come in different flavours. There differences are unimportant.

9 formal models of computation 9 TM M 1 to test if an input (on the tape) belongs to B = {w # w | w { 0, 1 } * } M 1 checks if the contents of the tape consists of two identical strings of 0s & 1s separated by # –M 1 accepts inputs 0110#0110, 000#000, # –M 1 rejects inputs 1111#0000, 000#0000, 0001#1000 M 1 should work for any size of input (general) –There are infinitely many, so we cannot use cases How would you program this? –All you have is a tape, and a head to read/write it Turing Machine: an Example

10 formal models of computation 10 Turing Machine: an Example (Contd) Strategy for M 1 : –Zig-zag to corresponding places at both sides of # and check if they match –Mark (cross off) those places you have checked –If it crosses off all symbols, then everything matches and M 1 goes into an accept state –If it discovers a mismatch then it enters a reject state

11 formal models of computation 11 Turing Machine: an Example (Contd) M 1 = On input string S 1.Scan the input to be sure it contains a single # symbol; if not, reject. 2.Zig-zag across the tape to corresponding positions on either side of # to check if they contain the same symbol. –If they do not match, reject. –Cross off symbols as they are checked to keep track of things. 3.When all symbols to the left of # have been crossed off, check for any remaining symbols to the right of #. –If any symbols remain, reject; otherwise accept.

12 formal models of computation 12 Turing Machine: an Example (Contd) Snapshots of M 1 s tape during stages 2 and 3: x #x11000x #x11000xx #x11000 xx #xx1000 x # #011000xxxx... xx#xxxxxx Accept!!

13 formal models of computation 13 Formal Definition of Turing Machines Previous slides give a flavour of TMs, but not their details. We can describe TMs formally, similarly to what you did for FAs We shall not always use formal descriptions for TMs because these would tend to be quite long –but ultimately, our informal descriptions should be translatable into formal ones –so it is crucial to understand these formal descriptions

14 formal models of computation 14 Formal definition of Turing Machines The imperative model is formulated in terms of actions: –A TM is always in one of a specified number of states (these include the accept and reject states) –The action of the TM, at a given moment, depends on its state and on what the TM is reading at that moment –Always perform three types of actions: (1) replace the symbol that it reads by another (or the same) symbol, (2) move the head Left or Right, and (3) enter a new state (or the same state again).

15 formal models of computation 15 Formal Definition of TMs (Contd) The heart of a TM is a function mapping –A state s of the machine –The contents a of the tape where the head is onto –A new state t of the machine –A symbol b to be written on the tape (over a) –A movement L (left) or R (right) of the head That is, (s,a) = (t,b,L) When the machine is in state s and the head is over a cell containing a symbol a, then the machine writes the symbol b (replacing a), goes to state t and moves the head to the Left.

16 formal models of computation 16 Formal Definition of TMs (Contd) A Turing Machine is a 7-tuple (Q,,,, q 0, q acc, q rej ) where Q,, are all finite sets and 1. Q is the set of states 2. is the input alphabet not containing the special blank symbol 3. is the tape alphabet, where { } 4. : Q Q {L, R} is the transition function 5. q 0 Q is the start state 6. q acc Q is the accept state 7. q rej Q is the reject state, where q rej q acc

17 formal models of computation 17 How Turing Machines Compute M = (Q,,,, q 0, q acc, q rej ) M receives its input w = w 1 w 2 …w n * –w is on the leftmost n cells of the tape. –The rest of the tape is blank (infinite list of ) The head starts on the leftmost cell of the tape –NB: does not contain the, so the first blank on the tape marks the end of the input M follows the moves encoded in –If M tries to move its head to the left of the left-hand end of the tape, the head stays in the same place, even though the function indicates L –The computation continues until it enters either the accept or reject states, at which point M halts –If neither occurs, M goes on forever…

18 formal models of computation 18 Formalisation of TMs TMs use sudden death; FSAs can only accept a string once the string has been read in its entirety. –FSA accepts a string iff the string is the label of a successful path. –TMs accepts a string if, while processing it, an accept state is reached. From the above, several things might still be unclear –What exactly does it mean to move to the right? –What if the head is already at the rightmost edge? A bit of formalisation would be useful

19 formal models of computation 19 Configurations of TMs As a TM computes, changes occur in its: –State –Tape contents –Head location Configurations are represented in a special way: –When the TM is in state q, and –The contents of the tape is two strings uv, and –The head is on the leftmost position of string v –Then we represent this configuration as u q v Example: 1011 q These three items are a configuration of the TM q7q7

20 formal models of computation 20 Formalising TMs Computations Assume –a, b, c in (3 characters from the tape) –u and v in * (2 strings from the tape) –states q i and q j We can now say that (for all u and v) u a q i b v yields u q j a c v if (q i,b ) = (q j,c, L) Graphically: qiqi u... b av qjqj u... c av

21 formal models of computation 21 Formalising TMs Computations (Contd) A similar definition for rightward moves u a q i b v yields u a c q j v if (q i,b ) = (q j,c, R) Graphically: qiqi u... b av qjqj u... c av

22 formal models of computation 22 Formalising TMs Computations (Contd) Special cases when head at beginning of tape For the left-hand end, moving left: q i b v yields q j c v if (q i,b ) = (q j,c, L) –We prevent the head from falling off the left-hand end of the tape: For the left-hand end, moving right: q i b v yields c q j v if (q i,b ) = (q j,c, R)... c v qiqi... b v qjqj... c v qiqi... b v qjqj

23 formal models of computation 23 Formalising TMs Computations (Contd) For the right-hand end (not really the end…) –infinite sequence of blanks follows the part of the tape represented in the configuration We thus handle the case above as before

24 formal models of computation 24 Formalising TMs Computations (Contd) The start configuration of M on input w is q 0 w –M in start state q 0 with head at leftmost position of tape An accepting configuration has state q acc A rejecting configuration has state q rej Rejecting and accepting configurations are halting configurations –They do not yield further configurations –No matter what else is in the configuration! Note: is a function, so the TM (as defined here) is deterministic

25 formal models of computation 25 Formalising TMs Computations (Contd) A TM M accepts input w if there is a sequence of configurations C 1, C 2,…,C k where 1. C 1 is the start configuration of M on input w 2. Each C i yields C i +1, and 3. C k is an accepting configuration Analogous to FSAs, the set of strings that M accepts is the language of M, denoted L(M ) we also say that a TM accepts a language. –If we want to avoid ambiguity between the two uses of accepts, we say M recognises L.

26 formal models of computation 26 A very simple TM (diagram and formal) Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0}

27 formal models of computation 27 Some shorthands to simplify notation: 1. when in state q 1 with the head reading 0, it goes to state q 2, writes and moves the head to the right 2. machine moves its head to the right when reading 0 in the state q 3, but nothing is written onto the tape Sample Turing Machine (Contd) q1q1 q2q2 0,R (q 1, 0 ) = (q 2,, R) (q 3, 0 ) = (q 4, 0,R) q3q3 q4q4 0 R

28 formal models of computation 28 A very simple TM (diagram and formal) Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0} Acc Rej q0q0 1 R 0 R R

29 formal models of computation 29 A very simple TM (diagram and formal) Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0} Acc Rej q0q0 1 R 0 R R Using formal notation: d(q 0,1)=(q 0,1,R). d(q 0,0)=(Acc,0,R). d(q 0,)=(Rej,,R)

30 formal models of computation 30 How would you modify this to recognize L = {w: w is a bitstring and w contains at least two 0s}? Sample Turing Machine

31 formal models of computation 31 Given: w is a bitstring Construct TM that recognises L={w: w contains at least two 0s} Acc Rej q1q1 1 R 0 R R q 0 1 R 0 R R

32 formal models of computation 32 More sophisticated TMs get complicated! –Very verbose and detailed document! –We often settle for a higher-level description –Easier to understand than transition rules or diagrams Important: every higher-level description is just a shorthand for its formal specification With patience and care it is possible to formally specify any TM given in a higher-level description Sample Turing Machine

33 formal models of computation 33 M 2 recognises all strings of 0s whose length is a power of 2, that is, the language A = { 0 2 n | n 0} Sample Turing Machine (Contd) M 2 = On input string w : 1. Sweep left to right across the tape, crossing off every other 0. If tape contains a single 0, accept If tape contains more than a single 0 and the number of 0 s was odd, reject. 2. Return the head to the left-hand of the tape 3. Go to step 1 = {0,00,0000, , ,...}

34 formal models of computation 34 Formally M 2 = (Q,,,, q 1, q acc, q rej ) Q = {q 1, q 2, q 3, q 4, q 5, q acc, q rej } = { 0 } = { 0, x, } is given as a state diagram (next slide) The start, accept and reject states are q 1, q acc, q rej Sample Turing Machine (Contd)

35 formal models of computation 35 s state diagram Sample Turing Machine (Contd) q1q1 q2q2 q5q5 q3q3 q rej q acc q4q4 R x R R 0,R 0 x,R x L 0 L R 0 R 0 x,R R L

36 formal models of computation 36 Sample run of M 2 on input 0000 : Sample Turing Machine (Contd) q q x q 3 00 x0 q 4 0 x0x q 3 x0 q 5 x x q 5 0x q 5 x0x x q 2 0x xx q 3 x xxx q 3 xx q 5 x q 2 x0x x q 5 xx q 5 xxx q 2 xxx x q 2 xx xx q 2 x xxx q 2 xxx q acc

37 formal models of computation 37 s state diagram Sample Turing Machine (Contd) q1q1 q2q2 q5q5 q3q3 q rej q acc q4q4 R x R R 0,R 0 x,R x L 0 L R 0 R 0 x,R R L

38 formal models of computation 38 A language is Turing-recognisable if some TM recognises it –Also called recursively enumerable language So if a TM recognises L, this means that all and only the elements of L are accepted by TM –these result in the Accept state But what happens with the strings that are not in L? Turing-Recognisable Languages

39 formal models of computation 39 A language is Turing-recognisable if some TM recognises it –Also called recursively enumerable language So if a TM recognises L, this means that all and only the elements of L are accepted by TM –these result in the Accept state But what happens with the strings s that are not in L? –these might end in the Reject state... –or the TM might never reach Accept or Reject on the input s Turing-Recognisable Languages

40 formal models of computation 40 A TM fails to accept an input either by –Entering the q rej and rejecting the input –Looping It is not easy to distinguish a machine that is looping from one that is just taking a long time! –Loops can be complex, not just C i C j C i C j … We prefer TMs that halt on all inputs –They never loop! They are called deciders –A decider that recognises a language is said to decide that language –A TM which is a decider answers every question (of the form Does the TM accept this string?) in finite time –Non-deciders keep you guessing on some strings Turing-Recognisable Languages (Contd)

41 formal models of computation 41 A language is Turing-decidable (or just decidable) if some TM decides it –Every decidable language is Turing-recognisable –Some Turing-recognisable languages are not decidable Turing-Decidable Languages

42 formal models of computation 42 There are many alternative definitions of TMs –They are called variants of the Turing machine They all have the same power as the original TM Some of these deviate from our TMs in minor ways. For example, Some TMs put strings in between two infinite sequences of blanks. This avoids the need that sometimes exists to mark the start of the string Some TMs halt only when they reach a state from which no transitions are possible. Compare our TM: sudden death after reaching q acc, q rej Lets briefly look at some more interesting variants Variants of Turing Machines

43 formal models of computation 43 A TM with several tapes instead of just one Each tape has its own head for reading/writing –Initially, input appears on tape 1; other tapes blank Transition function caters for k different tapes: : Q k Q k {L, R} k The expression (q i,a 1,…,a k ) = (q j, b 1, …, b k, L, R, …, L ) Means –If in q i and heads 1 through k read a 1 through a k, –Then go to q j, write b 1 through b k and move heads to the left or right, as specified Multitape Turing Machines

44 formal models of computation 44 Theorem: Every multitape Turing machine has an equivalent single tape Turing machine. Proof: show how to convert a multitape TM M to an equivalent single tape TM S. (Sketch only!) Multitape Turing Machines (Contd) M ba S aa a##ab ## a... aa

45 formal models of computation 45 This time, at a given point in the computation, there may be various possibilities to proceed Transition function maps to powerset (options): : Q 2 (Q {L, R}) The expression (q i,a ) = {(q j, b 1, L ),..., (q j, b 2, R )} Means –If in q i and head is on a –Then go to any one of the options (some finite number!) If there is one sequence of choices that leads to the accept state, the machine accepts the input Nondeterministic Turing Machines

46 formal models of computation 46 Theorem: For every nondeterministic Turing machine there exists an equivalent deterministic Turing machine Equivalent: -- accept the same strings -- reject the same strings (No proof offered here.) Nondeterministic TMs (Contd)

47 formal models of computation 47 These lectures focus on TMs for accepting strings (recognizing languages) Other TMs were built primarily for manipulating strings. Examples: –TM that count the number of symbols in a string (by producing a string that contains the right number of 1s) –TM that adds up two bit strings (by producing a string that contains the right number of 1s) –See your Practical Such TMs do not need Accept/Reject states. Their behaviour can be simulated in our TMs: accept if the tape contains inputs + the correct output. Other types of TMs

48 formal models of computation 48 Informally, an algorithm is –A sequence of instructions to carry out some task –A procedure or a recipe Algorithms have had a long history in maths: –Find prime numbers –Find greatest common divisors (Euclid, ca. 300 b.C.!) Algorithm was not defined until recently (1900s) –Before that, people had an intuitive idea of algorithm –This intuitive notion was insufficient to gain a better understanding of algorithms Next: how the precise notion of algorithm was crucial to an important mathematical problem… The Definition of Algorithm

49 formal models of computation 49 David Hilbert gave a lecture –1900 –Intl Congress of Mathematicians, Paris –Proposal of 23 mathematical problems –Challenges for the coming century –10 th problem concerned algorithms Before we talk about Hilberts 10 th problem, lets briefly discuss polynomials… Hilberts Problems

50 formal models of computation 50 A polynomial is a sum of terms, where each term is a product of variables (which may be exponentiated by a natural number) and a constant called a coefficient –For example: 6x 3 y z 2 + 3x y 2 – x 3 – 10 A root of a polynomial –Assign values to variables so that the polynomial is 0 –For example, if x = 5, y = 3 and z = 0, then ( ) + ( ) – 5 3 – 10 = 0 –This is an integral root: all variables assigned integers –Some polynomials have an integral root, some do not x 2 has integral root x 2 -3 has no integral root Polynomials

51 formal models of computation 51 Devise an algorithm that tests if a polynomial has an integral root –Hilbert wrote a process according to which it can be determined by a finite number of operations –Hilbert assumed such an algorithm existed: we only need to find it! We now know no algorithm exists for this task –It is algorithmically unsolvable: not computable Impossible to conclude this with only an intuitive notion of algorithm! –Proving that an algorithm does not exist requires a clear definition of algorithm –Hilberts 10 th Problem had to wait for this definition… Hilberts 10 th Problem

52 formal models of computation 52 The definition of algorithm came in 1936 –Papers by Alonzo Church and Alan Turing Church proposed a notational system – -Calculus (ha!) to define algorithms Turing proposed his abstract machines These two definitions were shown to be equivalent: Church-Turing thesis Thesis enabled a solution of Hilberts 10 th problem –Matijasevic (1970) showed no such algorithm exists Church-Turing Thesis

53 formal models of computation 53 Hilberts 10 th Problem in terms of TMs Consider the set D = { p | p is a polynomial with an integral root} Is D decidable? –Is there some TM that decides it? The answer is NO! However, D is Turing-recognisable To prove this, all we need to do is to supply a TM which does the deed: –The TM will halt if we input a polynomial that belongs to D, but it may loop if the polynomial does not belong to D Back to Hilberts 10 th Problem…

54 formal models of computation 54 Polynomials with only one variable: 4x 3 – 2x 2 + x – 7 Let D 1 = { p | p is a polynomial over x with an integral root} A TM M 1 that recognises D 1 : A simpler version of the 10 th Problem M 1 = The input is a polynomial p over x. 1. Evaluate p with x set successively to the values 0, 1, -1, 2, -2, 3, -3, … If at any point the polynomial evaluates to 0, accept!

55 formal models of computation 55 If p has an integral root, M 1 will eventually find the root and accept p If p does not have an integral root, M 1 will run forever! Its like the problem of determining (e.g. in Haskell) whether a given infinite list contains 0. For the multivariable case, a similar TM M –M goes through all possible integer values for each variable of the polynomial… Both M and M 1 are recognisers but not deciders Matijasevic showed 10 th problem has no decider This proof is omitted here. (But we shall prove another uncomputability result later.) A simpler version (Contd)

56 formal models of computation 56 The case with only 1 variable is decidable We can convert M 1 into a decider: –One can prove that the root x must lie between the values k (c max /c 1 ) where k is the number of terms in the polynomial c max is the coefficient with largest absolute value c 1 is the coefficient of the highest order term –We only have to try a finite number of values of x –If a root is not found within bounds, the machine rejects Matijasevic: impossible to calculate bounds for multivariable polynomials. Aside: Converting M 1 into a Decider

57 formal models of computation 57 We shall carry on talking about TMs However, our focus is on algorithms! –TM serves as a precise model for algorithms –We shall often make do with informal descriptions –We need to be comfortable enough with TMs to believe they capture all algorithms TMs vs. Algorithms

58 formal models of computation 58 Input to TM is always a string If we want to provide something else such as –Polynomial, matrix, list of students, etc. then we need to –Represent these as strings (somehow) AND –Program the TM to decode/act on the representation Notation: –Encoding of O as a string is O –Encoding of O 1, O 2,…, O k as a string is O 1, O 2,…, O k Notation for TMs

59 formal models of computation 59 Reading List Introduction to the Theory of Computation. Michael Sipser. PWS Publishing Co., USA, (A 2 nd Edition has recently been published). Chapter 3. Algorithmics: The Spirit of Computing. 3 rd Edition. David Harel with Yishai Feldman. Addison-Wesley, USA, Chapter 9.

60 formal models of computation 60 TM: Additional example

61 formal models of computation 61 Given: w is a bitstring Construct TM that recognises L={w: w contains exactly one 0} Acc Rej q1q1 1 R R 0 R q 0 1 R 0 R R

Download ppt "CS4026 Formal Models of Computation Part III Computability & Complexity Part III-A – Computability Theory."

Similar presentations

Ads by Google