Formal Models of Computation Part III Computability & Complexity

Formal Models of Computation Part III Computability & Complexity
Part III-A – Computability Theory

Formal models of computation
Before studying Haskell, we briefly examined the lambda calculus, a formal model underlying Haskell We shall soon explore another model of computation (called the imperative model), which underlies languages such as FORTRAN, PASCAL, C, JAVA etc: Turing Machines Let’s step back a second: Why study formal models (instead of only specific implementations)? formal models of computation

Why study formal models of comp.?
We do this to know what it means for a problem to be solvable by means of computation Which problems are “computable” in a given type of programming language? Best addressed by looking at a small prog. language As it happens, the answer does not depend on the type of programming language (functional / logical / imperative) The equivalence of various models of computation is known as Church’s Thesis This course: address computability by focussing on the imperative model (i.e., Turing Machines) formal models of computation

The Church-Turing Thesis
Overview (Chapter 3 of Sipser’s Book) Turing Machines Variants of Turing Machines Original motivation – find answers to these questions: What is an algorithm? What basic operations should algorithms contain? formal models of computation

formal models of computation
Turing Machines Abstract but accurate model of computers Proposed by Alan Turing in 1936 There weren’t computers back then! Turing’s motivation: find out whether there exist mathematical problems that cannot be solved algorithmically. (This is Hilbert’s “Entscheidungsproblem”, i.e., decision problem) Similar to a FSA but with an Unlimited and unrestricted memory Able to do everything a real computer can do However: There are problems even a Turing Machine (TM) can’t solve! Such problems are beyond the theoretical limits of computation formal models of computation

Turing Machines: Features
An tape is a countably infinite list of cells A tape head can read and write symbols from/onto a cell on the tape move forward and backward on the tape Schematically: Control Head a b □ . . . Tape formal models of computation

Turing Machines: Features (Cont’d)
Initially the tape Contains only the input string Is blank everywhere else If the TM needs to store information It can write it onto the tape To read the info it has written TM can move the head back over it TMs have the following behaviours: They “compute” then stop in a “reject” state They “compute” then stop in an “accept” state They loop forever Compare FSAs: These have no “reject” states, and no loops. (Why no loops?) formal models of computation

Turing Machines vs. Finite Automata
TMs can both write on the tape and read from it FSAs can only read (or only write if they are generators) The read/write head can move to the left and right FSAs cannot “go back” on the input The tape is infinite In FSAs the input and output is always finite The accept/reject states take immediate effect There is no need to “consume” all the input TMs come in different flavours. There differences are unimportant. formal models of computation

Turing Machine: an Example
TM M1 to test if an input (on the tape) belongs to B = {w #w | w  {0,1}* } M1 checks if the contents of the tape consists of two identical strings of 0’s & 1’s separated by “#” M1 accepts inputs 0110#0110, 000#000, # M1 rejects inputs 1111#0000, 000#0000, 0001#1000 M1 should work for any size of input (general) There are infinitely many, so we cannot use “cases” How would you program this? All you have is a tape, and a head to read/write it formal models of computation

Turing Machine: an Example (Cont’d)
Strategy for M1: “Zig-zag” to corresponding places at both sides of “#” and check if they match Mark (cross off) those places you have checked If it crosses off all symbols, then everything matches and M1 goes into an accept state If it discovers a mismatch then it enters a reject state formal models of computation

M1 = “On input string S Scan the input to be sure it contains a single # symbol; if not, reject. Zig-zag across the tape to corresponding positions on either side of # to check if they contain the same symbol. If they do not match, reject. Cross off symbols as they are checked to keep track of things. When all symbols to the left of # have been crossed off, check for any remaining symbols to the right of #. If any symbols remain, reject; otherwise accept.” formal models of computation

Snapshots of M1’s tape during stages 2 and 3: 1 □ . . . # x 1 □ . . . # x 1 □ . . . # x 1 □ . . . # x 1 □ . . . # x 1 □ . . . # . . . x □ . . . # Accept!! formal models of computation

Formal Definition of Turing Machines
Previous slides give a flavour of TMs, but not their details. We can describe TMs formally, similarly to what you did for FAs We shall not always use formal descriptions for TMs because these would tend to be quite long but ultimately, our informal descriptions should be “translatable” into formal ones so it is crucial to understand these formal descriptions formal models of computation

Formal definition of Turing Machines
The imperative model is formulated in terms of actions: A TM is always in one of a specified number of states (these include the accept and reject states) The action of the TM, at a given moment, depends on its state and on what the TM is reading at that moment Always perform three types of actions: (1) replace the symbol that it reads by another (or the same) symbol, (2) move the head Left or Right, and (3) enter a new state (or the same state again). formal models of computation

Formal Definition of TMs (Cont’d)
The heart of a TM is a function  mapping A state s of the machine The contents a of the tape where the head is onto A new state t of the machine A symbol b to be written on the tape (over a) A movement L (left) or R (right) of the head That is, (s,a) = (t,b,L) “When the machine is in state s and the head is over a cell containing a symbol a, then the machine writes the symbol b (replacing a), goes to state t and moves the head to the Left.” formal models of computation

Formal Definition of TMs (Cont’d)
A Turing Machine is a 7-tuple (Q, , , , q0, qacc , qrej ) where Q, ,  are all finite sets and Q is the set of states  is the input alphabet not containing the special blank symbol “□”  is the tape alphabet, where {□} : Q    Q    {L, R} is the transition function q0  Q is the start state qacc  Q is the accept state qrej  Q is the reject state, where qrej  qacc formal models of computation

How Turing Machines “Compute”
M = (Q, , , , q0, qacc , qrej ) M receives its input w = w1w2…wn  * w is on the leftmost n cells of the tape. The rest of the tape is blank (infinite list of “□”) The head starts on the leftmost cell of the tape NB:  does not contain the “□”, so the first blank on the tape marks the end of the input M follows the “moves” encoded in  If M tries to move its head to the left of the left-hand end of the tape, the head stays in the same place, even though the function indicates L The computation continues until it enters either the accept or reject states, at which point M halts If neither occurs, M goes on forever… n suggests that w should be finite. Should it? formal models of computation

Formalisation of TMs TMs use “sudden death”; FSAs can only accept a string once the string has been read in its entirety. FSA accepts a string iff the string is the label of a successful path. TMs accepts a string if, while processing it, an accept state is reached. From the above, several things might still be unclear What exactly does it mean to “move to the right”? What if the head is already at the rightmost edge? A bit of formalisation would be useful formal models of computation

Configurations of TMs As a TM computes, changes occur in its: State Tape contents Head location Configurations are represented in a special way: When the TM is in state q, and The contents of the tape is two strings uv, and The head is on the leftmost position of string v Then we represent this configuration as “u q v ” Example: 1011q These three items are a configuration of the TM 1 □ . . . q7 formal models of computation

Formalising TMs Computations
Assume a, b, c in  (3 characters from the tape) u and v in * (2 strings from the tape) states qi and qj We can now say that (for all u and v) u a qi b v yields u qj a c v if (qi ,b ) = (qj ,c , L) Graphically: qi u □ . . . b a v qj u □ . . . c a v formal models of computation

Formalising TMs Computations (Cont’d)
A similar definition for rightward moves u a qi b v yields u a c qj v if (qi ,b ) = (qj ,c , R) Graphically: qi u □ . . . b a v qj u □ . . . c a v formal models of computation

Special cases when head at beginning of tape For the left-hand end, moving left: qi b v yields qj c v if (qi ,b ) = (qj ,c , L) We prevent the head from “falling off” the left-hand end of the tape: For the left-hand end, moving right: qi b v yields c qj v if (qi ,b ) = (qj ,c , R) □ . . . c v qi b qj □ . . . c v qi b qj formal models of computation

For the right-hand “end” (not really the end…) infinite sequence of blanks follows the part of the tape represented in the configuration We thus handle the case above as before formal models of computation

The start configuration of M on input w is q0w M in start state q0 with head at leftmost position of tape An accepting configuration has state qacc A rejecting configuration has state qrej Rejecting and accepting configurations are halting configurations They do not yield further configurations No matter what else is in the configuration! Note:  is a function, so the TM (as defined here) is deterministic formal models of computation

A TM M accepts input w if there is a sequence of configurations C 1, C 2,…,C k where 1. C 1 is the start configuration of M on input w 2. Each C i yields C i +1, and 3. C k is an accepting configuration Analogous to FSAs, the set of strings that M accepts is the language of M, denoted L(M ) we also say that a TM “accepts” a language. If we want to avoid ambiguity between the two uses of “accepts”, we say “M recognises L”. formal models of computation

A very simple TM (diagram and formal)
Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0} formal models of computation

Sample Turing Machine (Cont’d)
Some shorthands to simplify notation: “when in state q1 with the head reading 0, it goes to state q2, writes □ and moves the head to the right” “machine moves its head to the right when reading 0 in the state q3, but nothing is written onto the tape” q1 q2 0  □,R (q1,0) = (q2, □, R) q3 q4 0  R (q3,0) = (q4,0,R) formal models of computation

Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0} 0 R q0 Acc 1R □R Rej formal models of computation

Given: w is a bitstring Construct TM that recognises L={w: w contains at least one 0} 0 R q0 Acc 1R □R Rej Using formal notation: d(q0,1)=(q0,1,R). d(q0,0)=(Acc,0,R). d(q0,□)=(Rej, □,R) formal models of computation

Sample Turing Machine How would you modify this to recognize L’ = {w: w is a bitstring and w contains at least two 0s}? formal models of computation

Given: w is a bitstring Construct TM that recognises L={w: w contains at least two 0s} 0  R q1 Acc 1R □ R 0  R Rej q0 □R 1  R formal models of computation

Sample Turing Machine More sophisticated TMs get complicated! Very verbose and detailed document! We often settle for a higher-level description Easier to understand than transition rules or diagrams Important: every higher-level description is just a shorthand for its formal specification With patience and care it is possible to formally specify any TM given in a higher-level description formal models of computation

M2 recognises all strings of 0s whose length is a power of 2, that is, the language A = { 02n | n  0} = {0,00,0000, , , ...} M2 = “On input string w : 1. Sweep left to right across the tape, crossing off every other 0. If tape contains a single 0, accept If tape contains more than a single 0 and the number of 0s was odd, reject. 2. Return the head to the left-hand of the tape 3. Go to step 1” formal models of computation

Formally M2 = (Q, , , , q1, qacc , qrej ) Q = {q1, q2, q3, q4, q5, qacc, qrej}  = {0}  = {0, x, □}  is given as a state diagram (next slide) The start, accept and reject states are q1, qacc , qrej formal models of computation

’s state diagram □  R x  R 0  □,R 0  x,R x  L 0  L 0  R □  L q1 q2 q5 q3 qrej qacc q4 formal models of computation

Sample run of M2 on input 0000: q10000 □q2000 □xq300 □x0q40 □x0xq3 □ □x0q5x □ □xq50x □ □q5x0x □ q5□x0x □ □xq20x □ □xxq3x □ □xxxq3 □ □xxq5x □ □q2x0x □ □xq5xx □ □q5xxx □ q5□xxx □ □q2xxx □ □xq2xx □ □xxq2x □ □xxxq2 □ □xxx □qacc formal models of computation

’s state diagram q1 q2 q5 q3 qrej qacc q4 □  R x  R 0  □,R 0  x,R x  L 0  L 0  R □  L formal models of computation

Turing-Recognisable Languages
A language is Turing-recognisable if some TM recognises it Also called recursively enumerable language So if a TM recognises L, this means that all and only the elements of L are accepted by TM these result in the Accept state But what happens with the strings that are not in L? formal models of computation

Turing-Recognisable Languages
A language is Turing-recognisable if some TM recognises it Also called recursively enumerable language So if a TM recognises L, this means that all and only the elements of L are accepted by TM these result in the Accept state But what happens with the strings s that are not in L? these might end in the Reject state ... or the TM might never reach Accept or Reject on the input s formal models of computation

Turing-Recognisable Languages (Cont’d)
A TM fails to accept an input either by Entering the qrej and rejecting the input Looping It is not easy to distinguish a machine that is looping from one that is just taking a long time! Loops can be complex, not just C i  C j  C i  C j  … We prefer TMs that halt on all inputs They never loop! They are called deciders A decider that recognises a language is said to decide that language A TM which is a decider answers every question (of the form “Does the TM accept this string?”) in finite time Non-deciders keep you guessing on some strings formal models of computation

Turing-Decidable Languages
A language is Turing-decidable (or just decidable) if some TM decides it Every decidable language is Turing-recognisable Some Turing-recognisable languages are not decidable formal models of computation

Variants of Turing Machines
There are many alternative definitions of TMs They are called variants of the Turing machine They all have the same power as the original TM Some of these deviate from “our” TMs in minor ways. For example, Some TMs put strings in between two infinite sequences of blanks. This avoids the need that sometimes exists to mark the start of the string Some TMs halt only when they reach a state from which no transitions are possible. Compare our TM: “sudden death” after reaching qacc , qrej Lets briefly look at some more interesting variants formal models of computation

Multitape Turing Machines
A TM with several tapes instead of just one Each tape has its own head for reading/writing Initially, input appears on tape 1; other tapes blank Transition function caters for k different tapes: : Q  k  Q  k  {L, R}k The expression (qi ,a1 ,…,ak ) = (qj , b1 , …, bk , L, R, …, L ) Means If in qi and heads 1 through k read a1 through ak , Then go to qj , write b1 through bk and move heads to the left or right, as specified formal models of computation

Multitape Turing Machines (Cont’d)
Theorem: “Every multitape Turing machine has an equivalent single tape Turing machine”. Proof: show how to convert a multitape TM M to an equivalent single tape TM S. (Sketch only!) M S . . . □ 1 . . . □ b a a . . . □ . . . □ 1 1 a a # b formal models of computation

Nondeterministic Turing Machines
This time, at a given point in the computation, there may be various possibilities to proceed Transition function maps to powerset (options): : Q    2(Q    {L, R}) The expression (qi ,a ) = {(qj , b1 , L ), ..., (qj , b2 , R )} Means If in qi and head is on a Then go to any one of the options (some finite number!) If there is one sequence of choices that leads to the accept state, the machine accepts the input formal models of computation

Nondeterministic TMs (Cont’d)
Theorem: “For every nondeterministic Turing machine there exists an equivalent deterministic Turing machine” “Equivalent”: -- accept the same strings -- reject the same strings (No proof offered here.) formal models of computation

Other types of TMs These lectures focus on TMs for accepting strings (recognizing languages) Other TMs were built primarily for manipulating strings. Examples: TM that count the number of symbols in a string (by producing a string that contains the right number of 1’s) TM that adds up two bit strings (by producing a string that contains the right number of 1’s) See your Practical Such TMs do not need Accept/Reject states. Their behaviour can be simulated in “our” TMs: accept if the tape contains inputs + the correct output. formal models of computation

The Definition of Algorithm
Informally, an algorithm is A sequence of instructions to carry out some task A procedure or a “recipe” Algorithms have had a long history in maths: Find prime numbers Find greatest common divisors (Euclid, ca. 300 b.C.!) “Algorithm” was not defined until recently (1900’s) Before that, people had an “intuitive” idea of algorithm This intuitive notion was insufficient to gain a better understanding of algorithms Next: how the precise notion of algorithm was crucial to an important mathematical problem… formal models of computation

Hilbert’s Problems David Hilbert gave a lecture 1900 Int’l Congress of Mathematicians, Paris Proposal of 23 mathematical problems Challenges for the coming century 10th problem concerned algorithms Before we talk about Hilbert’s 10th problem, let’s briefly discuss polynomials… formal models of computation

Polynomials A polynomial is a sum of terms, where each term is a product of variables (which may be exponentiated by a natural number) and a constant called a coefficient For example: 6x 3y z x y 2 – x 3 – 10 A root of a polynomial Assign values to variables so that the polynomial is 0 For example, if x = 5, y = 3 and z = 0, then (6  53  3  0 2) + (3  5  32) – 5 3 – 10 = 0 This is an integral root: all variables assigned integers Some polynomials have an integral root, some do not x2 has integral root x2-3 has no integral root formal models of computation

Hilbert’s 10th Problem Devise an algorithm that tests if a polynomial has an integral root Hilbert wrote “a process according to which it can be determined by a finite number of operations” Hilbert assumed such an algorithm existed: we only need to find it! We now know no algorithm exists for this task It is algorithmically unsolvable: not computable Impossible to conclude this with only an intuitive notion of algorithm! Proving that an algorithm does not exist requires a clear definition of algorithm Hilbert’s 10th Problem had to wait for this definition… formal models of computation

Church-Turing Thesis The definition of algorithm came in 1936 Papers by Alonzo Church and Alan Turing Church proposed a notational system -Calculus (ha!) to define algorithms Turing proposed his abstract machines These two definitions were shown to be equivalent: Church-Turing thesis Thesis enabled a solution of Hilbert’s 10th problem Matijasevic (1970) showed no such algorithm exists formal models of computation

Back to Hilbert’s 10th Problem…
Hilbert’s 10th Problem in terms of TMs Consider the set D = { p | p is a polynomial with an integral root} Is D decidable? Is there some TM that decides it? The answer is NO! However, D is Turing-recognisable To prove this, all we need to do is to supply a TM which “does the deed”: The TM will halt if we input a polynomial that belongs to D, but it may loop if the polynomial does not belong to D formal models of computation

A simpler version of the 10th Problem
Polynomials with only one variable: 4x 3 – 2x x – 7 Let D1= { p | p is a polynomial over x with an integral root} A TM M1 that recognises D1: M1 = “The input is a polynomial p over x. 1. Evaluate p with x set successively to the values 0, 1, -1, 2, -2, 3, -3, … If at any point the polynomial evaluates to 0, accept!” formal models of computation

A simpler version (Cont’d)
If p has an integral root, M1 will eventually find the root and accept p If p does not have an integral root, M1 will run forever! It’s like the problem of determining (e.g. in Haskell) whether a given infinite list contains 0. For the multivariable case, a similar TM M M goes through all possible integer values for each variable of the polynomial… Both M and M1 are recognisers but not deciders Matijasevic showed 10th problem has no decider This proof is omitted here. (But we shall prove another uncomputability result later.) formal models of computation

Aside: Converting M1 into a Decider
The case with only 1 variable is decidable We can convert M1 into a decider: One can prove that the root x must lie between the values k (cmax /c1 ) where k is the number of terms in the polynomial cmax is the coefficient with largest absolute value c is the coefficient of the highest order term We only have to try a finite number of values of x If a root is not found within bounds, the machine rejects Matijasevic: impossible to calculate bounds for multivariable polynomials. formal models of computation

TMs vs. Algorithms We shall carry on talking about TMs However, our focus is on algorithms! TM serves as a precise model for algorithms We shall often make do with informal descriptions We need to be comfortable enough with TMs to believe they capture all algorithms formal models of computation

Notation for TMs Input to TM is always a string If we want to provide something else such as Polynomial, matrix, list of students, etc. then we need to Represent these as strings (somehow) AND Program the TM to decode/act on the representation Notation: Encoding of O as a string is O  Encoding of O1, O2 ,… , Ok as a string is O1, O2 ,… , Ok  formal models of computation

Reading List Introduction to the Theory of Computation. Michael Sipser. PWS Publishing Co., USA, (A 2nd Edition has recently been published). Chapter 3. Algorithmics: The Spirit of Computing. 3rd Edition. David Harel with Yishai Feldman. Addison-Wesley, USA, Chapter 9. formal models of computation

TM: Additional example
formal models of computation

Given: w is a bitstring Construct TM that recognises L={w: w contains exactly one 0} □  R q1 Acc 1R 0R 0  R Rej q0 □R 1  R formal models of computation

Formal Models of Computation Part III Computability & Complexity

Similar presentations

Presentation on theme: "Formal Models of Computation Part III Computability & Complexity"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Formal Models of Computation Part III Computability & Complexity

Similar presentations

Presentation on theme: "Formal Models of Computation Part III Computability & Complexity"— Presentation transcript:

Similar presentations

About project

Feedback