Download presentation

Presentation is loading. Please wait.

Published byZander Lark Modified over 2 years ago

1
CS4026 Formal Models of Computation Part III Computability & Complexity Part III-A – Computability Theory

2
formal models of computation 2 Formal models of computation We have explored two formal models of computation (and an implementation of each): –Functions (Haskell) –Logic (Prolog) We shall soon explore a third model (called the imperative model), which underlies languages such as FORTRAN, PASCAL, C, JAVA etc: –Turing Machines Let’s step back a second: Why study formal models (instead of only specific implementations)?

3
formal models of computation 3 Why study formal models of comp.? We do this to know what it means for a problem to be solvable by means of computation –Which problems are “computable” in a given type of programming language? –Best addressed by looking at a small prog. language As it happens, the answer does not depend on the type of programming language (functional / logical / imperative) –The equivalence of various models of computation is known as Church’s Thesis This course: address computability by focussing on the imperative model (i.e., Turing Machines)

4
formal models of computation 4 The Church-Turing Thesis Overview (Chapter 3 of Sipser’s Book) Turing Machines Variants of Turing Machines Original motivation – find answers to these questions: What is an algorithm? What basic operations should algorithms contain?

5
formal models of computation 5 Turing Machines Abstract but accurate model of computers Proposed by Alan Turing in 1936 –There weren’t computers back then! Turing’s motivation: find out whether there exist mathematical problems that cannot be solved algorithmically. (This is Hilbert’s “Entscheidungsproblem”, i.e., decision problem) Similar to a finite automaton but with an –Unlimited and unrestricted memory Able to do everything a real computer can do However: –There are problems even a Turing Machine (TM) can’t solve! –Such problems are beyond the theoretical limits of computation

6
formal models of computation 6 Turing Machines: Features An tape is an infinite list of cells A tape head can –read and write symbols from/onto a cell on the tape –move forward and backward on the tape Schematically: Control Head abaa □□□... Tape

7
formal models of computation 7 Turing Machines: Features (Cont’d) Initially the tape –Contains only the input string –Is blank everywhere else If the TM needs to store information –It can write it onto the tape To read the info it has written –TM can move the head back over it TMs have the following behaviours: –They “compute” then stop in a “reject” state –They “compute” then stop in an “accept” state –They loop forever

8
formal models of computation 8 Turing Machines vs. Finite Automata TMs can both write on the tape and read from it –FAs can only read (or only write if they are generators) The read/write head can move to the left and right –FAs cannot “go back” on the input The tape is infinite –In FAs the input and output is always finite The accept/reject states take immediate effect –There is no need to “consume” all the input TMs come in different flavours. There differences are unimportant.

9
formal models of computation 9 TM M 1 to test if an input (on the tape) belongs to B = {w # w | w { 0, 1 } * } M 1 checks if the contents of the tape consists of two identical strings of 0’s & 1’s separated by “#” –M 1 accepts inputs 0110#0110, 000#000, 010101010#010101010 –M 1 rejects inputs 1111#0000, 000#0000, 0001#1000 M 1 should work for any size of input (general) –There are infinitely many, so we cannot use “cases” How would you program this? –All you have is a tape, and a head to read/write it Turing Machine: an Example

10
formal models of computation 10 Turing Machine: an Example (Cont’d) Strategy for M 1 : –“Zig-zag” to corresponding places at both sides of “ # ” and check if they match –Mark (cross off) those places you have checked –If it crosses off all symbols, then everything matches and M 1 goes into an accept state –If it discovers a mismatch then it enters a reject state

11
formal models of computation 11 Turing Machine: an Example (Cont’d) M 1 = “On input string S 1.Scan the input to be sure it contains a single # symbol; if not, reject. 2.Zig-zag across the tape to corresponding positions on either side of # to check if they contain the same symbol. –If they do not match, reject. –Cross off symbols as they are checked to keep track of things. 3.When all symbols to the left of # have been crossed off, check for any remaining symbols to the right of #. –If any symbols remain, reject; otherwise accept.”

12
formal models of computation 12 Turing Machine: an Example (Cont’d) Snapshots of M 1 ’s tape during stages 2 and 3: x110 □... 00#x11000x110 □ 00#x11000xx10 □ 00#x11000 xx10 □ 00#xx1000 x110 □ 00#0110000110 □ 00#011000xxxx □ xx#xxxxxx Accept!!

13
formal models of computation 13 Formal Definition of Turing Machines Previous slides give a flavour of TMs, but not their details. We can describe TMs formally, similarly to what you did for FAs We shall not always use formal descriptions for TMs because these would tend to be quite long –but ultimately, our informal descriptions should be “translatable” into formal ones –so it is crucial to understand these formal descriptions

14
formal models of computation 14 Formal definition of Turing Machines The imperative model is formulated in terms of actions: –A TM is always in one of a specified number of states (these include the accept and reject states) –The action of the TM, at a given moment, depends on its state and on what the TM is reading at that moment –Always perform three types of actions: (1) replace the symbol that it reads by another (or the same) symbol, (2) move the head Left or Right, and (3) enter a new state (or the same state again).

15
formal models of computation 15 Formal Definition of TMs (Cont’d) The heart of a TM is a function mapping –A state s of the machine –The contents a of the tape where the head is onto –A new state t of the machine –A symbol b to be written on the tape (over a) –A movement L (left) or R (right) of the head That is, (s,a) = (t,b,L) “When the machine is in state s and the head is over a cell containing a symbol a, then the machine writes the symbol b (replacing a), goes to state t and moves the head to the Left.”

16
formal models of computation 16 Formal Definition of TMs (Cont’d) A Turing Machine is a 7-tuple (Q, , , , q 0, q acc, q rej ) where Q, , are all finite sets and 1. Q is the set of states 2. is the input alphabet not containing the special blank symbol “ □ ” 3. is the tape alphabet, where { □ } 4. : Q Q {L, R} is the transition function 5. q 0 Q is the start state 6. q acc Q is the accept state 7. q rej Q is the reject state, where q rej q acc

17
formal models of computation 17 How Turing Machines “Compute” M = (Q, , , , q 0, q acc, q rej ) M receives its input w = w 1 w 2 …w n * –w is on the leftmost n cells of the tape. –The rest of the tape is blank (infinite list of “ □ ”) The head starts on the leftmost cell of the tape –NB: does not contain the “ □ ”, so the first blank on the tape marks the end of the input M follows the “moves” encoded in –If M tries to move its head to the left of the left-hand end of the tape, the head stays in the same place, even though the function indicates L –The computation continues until it enters either the accept or reject states, at which point M halts –If neither occurs, M goes on forever…

18
formal models of computation 18 Formalisation of TMs From the above, several things might still be unclear –What exactly does it mean to “move to the right”? –What if the head is already at the rightmost edge? A bit of formalisation would be useful

19
formal models of computation 19 Configurations of TMs As a TM computes, changes occur in its: –State –Tape contents –Head location Configurations are represented in a special way: –When the TM is in state q, and –The contents of the tape is two strings uv, and –The head is on the leftmost position of string v –Then we represent this configuration as “u q v ” Example: 1011 q 7 011111 These three items are a configuration of the TM 1011 □... 011111 □□ q7q7

20
formal models of computation 20 Formalising TMs Computations Assume –a, b, c in (3 characters from the tape) –u and v in * (2 strings from the tape) –states q i and q j We can now say that (for all u and v) u a q i b v yields u q j a c v if (q i,b ) = (q j,c, L) Graphically: qiqi u □... b av qjqj u □ c av

21
formal models of computation 21 Formalising TMs Computations (Cont’d) A similar definition for rightward moves u a q i b v yields u a c q j v if (q i,b ) = (q j,c, R) Graphically: qiqi u □... b av qjqj u □ c av

22
formal models of computation 22 Formalising TMs Computations (Cont’d) Special cases when head at beginning of tape For the left-hand end, moving left: q i b v yields q j c v if (q i,b ) = (q j,c, L) –We prevent the head from “falling off” the left-hand end of the tape: For the left-hand end, moving right: q i b v yields c q j v if (q i,b ) = (q j,c, R) □... c v qiqi □ b v qjqj □ c v qiqi □ b v qjqj

23
formal models of computation 23 Formalising TMs Computations (Cont’d) For the right-hand “end” (not really the end…) –infinite sequence of blanks follows the part of the tape represented in the configuration We thus handle the case above as before

24
formal models of computation 24 Formalising TMs Computations (Cont’d) The start configuration of M on input w is q 0 w –M in start state q 0 with head at leftmost position of tape An accepting configuration has state q acc A rejecting configuration has state q rej Rejecting and accepting configurations are halting configurations –They do not yield further configurations –No matter what else is in the configuration! Note: is a function, so the TM (as defined here) is deterministic

25
formal models of computation 25 Formalising TMs Computations (Cont’d) A TM M accepts input w if there is a sequence of configurations C 1, C 2,…,C k where 1. C 1 is the start configuration of M on input w 2. Each C i yields C i +1, and 3. C k is an accepting configuration The set of strings that M accepts is the language of M, denoted L(M ) We also say that a TM “accepts” a language (rather than the strings in it). If we want to avoid ambiguity between the two uses of “accepts”, we say “M recognises L”.

26
formal models of computation 26 A language is Turing-recognisable if some TM recognises it –Also called recursively enumerable language So if a TM recognises L, this means that all and only the elements of L are accepted by TM –these result in the Accept state But what happens with the strings that are not in L? Turing-Recognisable Languages

27
formal models of computation 27 A language is Turing-recognisable if some TM recognises it –Also called recursively enumerable language So if a TM recognises L, this means that all and only the elements of L are accepted by TM –these result in the Accept state But what happens with the strings s that are not in L? –these might end in the Reject state... –or the TM might never reach Accept or Reject on the input s Turing-Recognisable Languages

28
formal models of computation 28 A TM fails to accept an input either by –Entering the q rej and rejecting the input –Looping It is not easy to distinguish a machine that is looping from one that is just taking a long time! –Loops can be complex, not just C i C j C i C j … We prefer TMs that halt on all inputs –They never loop! They are called deciders –A decider that recognises a language is said to decide that language –A TM which is a decider answers every question (of the form “Does the TM accept this string?”) in finite time –Non-deciders keep you guessing on some strings Turing-Recognisable Languages (Cont’d)

29
formal models of computation 29 A language is Turing-decidable (or just decidable) if some TM decides it –Every decidable language is Turing-recognisable –Some Turing-recognisable languages are not decidable Before exploring computability any further, let’s look at TMs more closely. Turing-Decidable Languages

30
formal models of computation 30 A very simple TM (diagram and formal) Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0}

31
formal models of computation 31 Some shorthands to simplify notation: 1. “when in state q 1 with the head reading 0, it goes to state q 2, writes □ and moves the head to the right” 2. “machine moves its head to the right when reading 0 in the state q 3, but nothing is written onto the tape” Sample Turing Machine (Cont’d) q1q1 q2q2 0 □,R (q 1, 0 ) = (q 2, □, R) (q 3, 0 ) = (q 4, 0,R) q3q3 q4q4 0 R

32
formal models of computation 32 A very simple TM (diagram and formal) Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0} Acc Rej q0q0 1R1R 0 R □R□R

33
formal models of computation 33 A very simple TM (diagram and formal) Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0} Acc Rej q0q0 1R1R 0 R □R□R Using formal notation: d(q 0,1)=(q 0,1,R). d(q 0,0)=(Acc,0,R). d(q 0,□)=(Rej, □,R)

34
formal models of computation 34 How would you modify this to recognize L’ = {w: w is a bitstring and w contains at least two 0s}? Sample Turing Machine

35
formal models of computation 35 Given: w is a bitstring Construct TM that recognises L={w: w contains at least two 0s} Acc Rej q1q1 1R1R 0 R □ R□ R q 0 1 R 0 R □R□R

36
formal models of computation 36 More sophisticated TMs get complicated! –Very verbose and detailed document! –We often settle for a higher-level description –Easier to understand than transition rules or diagrams Important: every higher-level description is just a shorthand for its formal specification With patience and care it is possible to formally specify any TM given in a higher-level description Sample Turing Machine

37
formal models of computation 37 M 2 recognises all strings of 0s whose length is a power of 2, that is, the language A = { 0 2 n | n 0} Sample Turing Machine (Cont’d) M 2 = “On input string w : 1. Sweep left to right across the tape, crossing off every other 0. If tape contains a single 0, accept If tape contains more than a single 0 and the number of 0 s was odd, reject. 2. Return the head to the left-hand of the tape 3. Go to step 1” = {0,00,0000,00000000,0000000000000000,...}

38
formal models of computation 38 Formally M 2 = (Q, , , , q 1, q acc, q rej ) Q = {q 1, q 2, q 3, q 4, q 5, q acc, q rej } = { 0 } = { 0, x, □ } is given as a state diagram (next slide) The start, accept and reject states are q 1, q acc, q rej Sample Turing Machine (Cont’d)

39
formal models of computation 39 ’s state diagram Sample Turing Machine (Cont’d) q1q1 q2q2 q5q5 q3q3 q rej q acc q4q4 □ R x R □ R x R 0 □,R 0 x,R x L 0 L □ R 0 R 0 x,R □ R □ L

40
formal models of computation 40 Sample run of M 2 on input 0000 : Sample Turing Machine (Cont’d) q 1 0000 □ q 2 000 □x q 3 00 □x0 q 4 0 □x0x q 3 □ □x0 q 5 x □ □x q 5 0x □ □ q 5 x0x □ q 5 □x0x □ □x q 2 0x □ □xx q 3 x □ □xxx q 3 □ □xx q 5 x □ □ q 2 x0x □ □x q 5 xx □ □ q 5 xxx □ q 5 □xxx □ □ q 2 xxx □ □x q 2 xx □ □xx q 2 x □ □xxx q 2 □ □xxx □ q acc

41
formal models of computation 41 ’s state diagram Sample Turing Machine (Cont’d) q1q1 q2q2 q5q5 q3q3 q rej q acc q4q4 □ R x R □ R x R 0 □,R 0 x,R x L 0 L □ R 0 R 0 x,R □ R □ L

42
formal models of computation 42 There are many alternative definitions of TMs –They are called variants of the Turing machine They all have the same power as the original TM Some of these deviate from “our” TMs in minor ways. For example, Some TMs put strings in between two infinite sequences of blanks. This avoids the need that sometimes exists to mark the start of the string Some TMs halt only when they reach a state from which no transitions are possible. Compare our TM: “sudden death” after reaching q acc, q rej Lets briefly look at some more interesting variants Variants of Turing Machines

43
formal models of computation 43 A TM with several tapes instead of just one Each tape has its own head for reading/writing –Initially, input appears on tape 1; other tapes blank Transition function caters for k different tapes: : Q k Q k {L, R} k The expression (q i,a 1,…,a k ) = (q j, b 1, …, b k, L, R, …, L ) Means –If in q i and heads 1 through k read a 1 through a k, –Then go to q j, write b 1 through b k and move heads to the left or right, as specified Multitape Turing Machines

44
formal models of computation 44 Theorem: “Every multitape Turing machine has an equivalent single tape Turing machine”. Proof: show how to convert a multitape TM M to an equivalent single tape TM S. (Sketch only!) Multitape Turing Machines (Cont’d) M 0... □101 □ba S 0 □ 11 01aaa a##abb ## a □aa

45
formal models of computation 45 This time, at a given point in the computation, there may be various possibilities to proceed Transition function maps to powerset (options): : Q 2 (Q {L, R}) The expression (q i,a ) = {(q j, b 1, L ),..., (q j, b 2, R )} Means –If in q i and head is on a –Then go to any one of the options (some finite number!) If there is one sequence of choices that leads to the accept state, the machine accepts the input Nondeterministic Turing Machines

46
formal models of computation 46 Theorem: “For every nondeterministic Turing machine there exists an equivalent deterministic Turing machine” “Equivalent”: -- accept the same strings -- reject the same strings (No proof offered here.) Nondeterministic TMs (Cont’d)

47
formal models of computation 47 These lectures focus on TMs for accepting strings (recognizing languages) Other TMs were built primarily for manipulating strings. Examples: –TM that count the number of symbols in a string (by producing a string that contains the right number of 1’s) –TM that adds up two bit strings (by producing a string that contains the right number of 1’s) –See your Practical Such TMs do not need Accept/Reject states. Their behaviour can be simulated in “our” TMs: accept if the tape contains inputs + the correct output. Other types of TMs

48
formal models of computation 48 Informally, an algorithm is –A sequence of instructions to carry out some task –A procedure or a “recipe” Algorithms have had a long history in maths: –Find prime numbers –Find greatest common divisors (Euclid, ca. 300 b.C.!) “Algorithm” was not defined until recently (1900’s) –Before that, people had an “intuitive” idea of algorithm –This intuitive notion was insufficient to gain a better understanding of algorithms Next: how the precise notion of algorithm was crucial to an important mathematical problem… The Definition of Algorithm

49
formal models of computation 49 David Hilbert gave a lecture –1900 –Int’l Congress of Mathematicians, Paris –Proposal of 23 mathematical problems –Challenges for the coming century –10 th problem concerned algorithms Before we talk about Hilbert’s 10 th problem, let’s briefly discuss polynomials… Hilbert’s Problems

50
formal models of computation 50 A polynomial is a sum of terms, where each term is a product of variables (which may be exponentiated by a natural number) and a constant called a coefficient –For example: 6x 3 y z 2 + 3x y 2 – x 3 – 10 A root of a polynomial –Assign values to variables so that the polynomial is 0 –For example, if x = 5, y = 3 and z = 0, then (6 5 3 3 0 2 ) + (3 5 3 2 ) – 5 3 – 10 = 0 –This is an integral root: all variables assigned integers –Some polynomials have an integral root, some do not x 2 has integral root x 2 -3 has no integral root Polynomials

51
formal models of computation 51 Devise an algorithm that tests if a polynomial has an integral root –Hilbert wrote “a process according to which it can be determined by a finite number of operations” –Hilbert assumed such an algorithm existed: we only need to find it! We now know no algorithm exists for this task –It is algorithmically unsolvable: not computable Impossible to conclude this with only an intuitive notion of algorithm! –Proving that an algorithm does not exist requires a clear definition of algorithm –Hilbert’s 10 th Problem had to wait for this definition… Hilbert’s 10 th Problem

52
formal models of computation 52 The definition of algorithm came in 1936 –Papers by Alonzo Church and Alan Turing Church proposed a notational system – -Calculus (ha!) to define algorithms Turing proposed his abstract machines These two definitions were shown to be equivalent: Church-Turing thesis Thesis enabled a solution of Hilbert’s 10 th problem –Matijasevic (1970) showed no such algorithm exists Church-Turing Thesis

53
formal models of computation 53 Hilbert’s 10 th Problem in terms of TMs Consider the set D = { p | p is a polynomial with an integral root} Is D decidable? –Is there some TM that decides it? The answer is NO! However, D is Turing-recognisable To prove this, all we need to do is to supply a TM which “does the deed”: –The TM will halt if we input a polynomial that belongs to D, but it may loop if the polynomial does not belong to D Back to Hilbert’s 10 th Problem…

54
formal models of computation 54 Polynomials with only one variable: 4x 3 – 2x 2 + x – 7 Let D 1 = { p | p is a polynomial over x with an integral root} A TM M 1 that recognises D 1 : A simpler version of the 10 th Problem M 1 = “The input is a polynomial p over x. 1. Evaluate p with x set successively to the values 0, 1, -1, 2, -2, 3, -3, … If at any point the polynomial evaluates to 0, accept!”

55
formal models of computation 55 If p has an integral root, M 1 will eventually find the root and accept p If p does not have an integral root, M 1 will run forever! It’s like the problem of determining (e.g. in Haskell) whether a given infinite list contains 0. For the multivariable case, a similar TM M –M goes through all possible integer values for each variable of the polynomial… Both M and M 1 are recognisers but not deciders Matijasevic showed 10 th problem has no decider This proof is omitted here. (But we shall prove another uncomputability result later.) A simpler version (Cont’d)

56
formal models of computation 56 The case with only 1 variable is decidable We can convert M 1 into a decider: –One can prove that the root x must lie between the values k (c max /c 1 ) where k is the number of terms in the polynomial c max is the coefficient with largest absolute value c 1 is the coefficient of the highest order term –We only have to try a finite number of values of x –If a root is not found within bounds, the machine rejects Matijasevic: impossible to calculate bounds for multivariable polynomials. Aside: Converting M 1 into a Decider

57
formal models of computation 57 We shall carry on talking about TMs However, our focus is on algorithms! –TM serves as a precise model for algorithms –We shall often make do with informal descriptions –We need to be comfortable enough with TMs to believe they capture all algorithms TMs vs. Algorithms

58
formal models of computation 58 Input to TM is always a string If we want to provide something else such as –Polynomial, matrix, list of students, etc. then we need to –Represent these as strings (somehow) AND –Program the TM to decode/act on the representation Notation: –Encoding of O as a string is O –Encoding of O 1, O 2,…, O k as a string is O 1, O 2,…, O k Notation for TMs

59
formal models of computation 59 Reading List Introduction to the Theory of Computation. Michael Sipser. PWS Publishing Co., USA, 1997. (A 2 nd Edition has recently been published). Chapter 3. Algorithmics: The Spirit of Computing. 3 rd Edition. David Harel with Yishai Feldman. Addison-Wesley, USA, 2004. Chapter 9.

60
formal models of computation 60 TM: Additional example

61
formal models of computation 61 Given: w is a bitstring Construct TM that recognises L={w: w contains exactly one 0} Acc Rej q1q1 1R1R □ R 0R0R q 0 1 R 0 R □R□R

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google