Donghyun (David) Kim Department of Mathematics and Computer Science North Carolina Central University 1 Chapter 3 Church-Turing Thesis Some slides are in courtesy of Prof. Verma at University of Houston Spring 2016 CISG 5115 Theory of Computation

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Universal Models of Computation Models of computation that have power equivalent to that of an idealized general-purpose computer or, models of computation characterize problem-solving by humans and machines. how is the input (output) represented? how does the computational model compute? what is the cost (in time and space) ? Use models to determine what can and cannot be computed. 2

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Issues of Computability “What problems can be solved on a given model of computation?” Halting Problem - a classical unsolvable problem: “Given a description of a program, decide whether the program will finish running or continue run forever.” Another one: “Does the program return the correct answer?” 3

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University A Contradiction Argument for Halting Problem Suppose there exists an algorithm P for the Halting Problem. boolean function P (string prog, data) { } P returns T (“prog" stops on “data") or F (“prog" doesn't stop) Now consider the following function Q. boolean function Q (string x) { if (P(x, x) == F) return F; while (1) ; } Q(x) ? Q(x) stops iff P(x,x) returns F, which happens iff P(x,x) doesn't stop on x. Q(x) doesn't stop iff P(x,x) returns T, which happens iff P(x,x) stops on x. Contradiction P (Q, Q) returns T => Q(Q) stops => P (Q,Q) does not stop and return F. P (Q, Q) returns F => Q(Q) does not stop => P (Q,Q) stops and return T. 4

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University 3.1 Turing Machines 5

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing Machine (TM) Proposed by Alan Turing in Mimic the problem-solving mechanism of a scientist unbounded supply of paper, pencils, erasers and thinks. write/lookup/modify notes a finite number of mental states: finite volume of brain and discrete thought processes. make decision based on notes and the scientist's current mental state TM: has an unlimited and unrestricted memory (an infinite tape). read/write tape move left/right rejecting/accepting states take immediate effect. Equivalent in power with a real computer. 6

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing Machine – cont’ 7... Finite State Control Bi-direction Read/Write Infinite tape (unlimited memory) Tape head (read/write) Initially, a tape contains only the input string and is blank everywhere else.

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing Machine – cont’ 8

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing Machine – cont’ We can design TM’s for computing functions from strings to strings We can design TM’s to decide languages using special states accept/reject or by writing Y/N on tape. We can design TM’s to accept languages. if TM halts at an accept status (after consuming the whole string), the string is accepted 9

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Example of TM for We want a Turing machine M to accept if its input is a member of B and to reject otherwise. Our goal is to determine whether the input is a member of B – whether the input comprises two identical strings separated by a # symbol. The input is too long for you to remember it all, but you are allowed to move back and forth over the input and make marks on it. 10

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Example of TM for – cont’ Design of M It makes multiple passes over the input string with the read-write head. On each pass, it matches one of the characters on each side of the #symbol. To keep track of which symbols have been checked already, M crosses off each symbol as it is examined. If it crosses off all the symbols, that means that everything matched successfully, and M goes into an accept state. Otherwise, reject #0110 X110#0110 cross off 0

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Example of TM for – cont’ Algorithm of M on input string w Zig-zag across the tape to corresponding positions on either side of the # symbol to check whether these positions contain the same symbol. If they do not, or if no # is found, reject. Cross off symbols as they are checked to keep track of which symbols correspond. When all symbols to the left of the # have been crossed off, check for any remaining symbols to the right of the #. If any symbols remain, reject; otherwise accept. 12

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Example of TM for – cont’ #0110 x110#0110 x110#x110 x110#x110 xx10#x110 xxxx#xxxx accept

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Formal Definition of a TM 14

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Operations of TM Copy input to tape. Start on the leftmost symbol on the tape. Proceed according to the transition rules. (Never move off the left end of the tape.) Continue until it enters the accept state, or the reject state, or goes on forever. 15

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Configurations of TM Configuration: a snap shot (3 items) current state current tape contents current head location e.g. on the tape, is current state, head is on 3 rd 0 Start configuration: Halting configurations (does not yield further configurations) accepting configuration: accept state rejecting configuration: reject state 16

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Language of TM Yield: configuration yields if the TM can go from to in one step. e.g. If, then yields A TM M accepts a string w : M goes from the start configuration to an accepting configuration on input w. The language of M (the language recognized by M ): 17

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing-recognizable Languages Definition: A language is Turing-recognizable (recursively enumerable, r.e.) if some Turing machine recognizes (accepts) it, i.e., A TM has three possible outcomes: accept, reject, or not halt (loop) If L is regular., we can determine if, but not necessarily if. Thus, we don't have an algorithm for determining whether or not. (not halt = either loop or still computing?) 18

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing-decidable Languages 19

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Example The description of TM that decides, the language consisting of all strings of 0s whose length is a power of 2. = “On input string w ” 1.Sweep left to right across the tape, crossing off (x) every other 0 2.If in stage 1 the tape contained a single 0, accept 3.If in stage 1 the tape contained more than a single 0 and the number of 0s was odd, reject 4.Return the head to the left-hand end of the tape 5.Go to stage 1 20

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University 3.2 Variants of Turing Machines 21

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing Machine: Variations A number of variations of TM possible such as: a TM with possible “stay put” moves of the head; a TM with a tape in finite in both directions; a TM with multiple tapes; a non-deterministic TM. It turns out that such variations have no more computation power than the “regular” TM as we defined it (equivalent!) – recognize the same class of languages To prove this, it is enough to show that there is a regular TM simulates a particular variation, i.e., they always produce the same result for the same input. 22

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Turing Machine: Simulation Note that configuration of a TM can be encoded by a (finite) string that other TM can read and write on its tape. Furthermore, the transition function of a TM can be also described by a (finite) string. These observations lead to idea of encoding and running the whole TM by another TM. 23

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Multi-tape TMs It is an ordinary Turing machine with several tapes. Initially, the input appears on tape 1, and the others start out blank. Several tapes, each has its own head for reading and writing. Theorem: Every multitape TM has an equivalent single tape TM. Proof. Simulate a multitape TM with a single tape TM: use one tape to represent all tapes 24

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Multi-tape TMs – cont’ 25 M 0101 aaa ba S #100#aa#a

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Nondeterministic TMs Nondeterministic computation | several possibilities to proceed Theorem: Every nondeterministic TM has an equivalent deterministic TM. Proof. Simulate a nondeterministic TM with a deterministic TM: try all possible derivations. 26

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Nondeterministic TMs – cont’ 27 D 0010 xx# 12 1x input tape (never altered) simulation tape (for a branch of nondeterministic computation) address tape (keeps track of D’s location in N’s nondeterministic computation tree)

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Enumerators A TM with an attached printer. (loosely speaking) Start with a blank input tape The language enumerated is all the strings it prints out. Theorem: A language is Turing-recognizable if and only if some enumerator enumerates it. M = on input w, 1.Run E. Every time that E outputs a string, compare it with w. 2.If w ever appears in the output of E, accept. Construct an enumerator E as follows : 1.Run M for each possible 2.If M accept s, print out 28

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University 3.3 The Definition of Algorithm 29

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Algorithms Algorithms have been around since ancient times. Informally, it is a collection of simple instructions for carrying out some task. They can be called “procedures” or “recipes”. One of the famous algorithms was given by Euclid around 300 BC (now called Euclid's algorithm) for finding the greatest common divisor (GCD) of two integers. However, there was no rigorous definition of the algorithm until the twentieth century 30

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Hilbert's Tenth Problem In 1900, at the International Congress of Mathematicians, David Hilbert posed a list of open problems as a challenge for the coming century. The tenth problem was about an algorithm. He asked to “devise” a procedure for answering to the question whether a given polynomial such as possess an integer root, i.e., a set of integer values of the variables at which the polynomial takes zero value (e.g., x = 5, y = 3, z = 0 is an integer root of the above polynomial). 31 coefficient term

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Hilbert's Tenth Problem – cont’ By asking to devise a procedural way to answer this question, Hilbert implicitly assumed that such a procedure (algorithm) exists. There was no way to come up with a conclusion without the definition of algorithm. 32

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Church-Turing thesis 33

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Church-Turing thesis – cont’ Hilbert's Tenth Problem basically asks to construct a decider for the language: That is, for any given polynomial p, we need to decide whether (in a finite number of steps). That is, Hilbert’s tenth problem asks in essence whether the set D is decidable. 34

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Example: Finding an integer root A Turing machine for finding an integer root of a given polynomial (of a single variable x for simplicity, i.e. ) can be informally described as: Evaluate a given polynomial p ( x ) at x set successively to the values 0, 1, -1, 2, -2, 3, -3, … If at any such value of x the value p ( x ) is zero, then accept. Q: Is this Turing machine recognizer or decider? 35

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Example: Finding an integer root Q: Is this Turing machine recognizer or decider? For a univariate polynomial, it is possible to compute an upper bound for an absolute value of its integer root. If no root found below this bound, the polynomial has no integer roots, and its TM can reject (making it a decider solving the problem). However, as shown by Matijasevic, for multivariate polynomials no such bound can be computed, making it impossible to have a decider for this problem in general case. 36

Spring 2016 CISG 5115 Department of Mathematics and Physics Donghyun (David) Kim North Carolina Central University Description of (Turing Machine) Algorithms Algorithms can be described in three ways: formal: Explicit description of a Turing machine (its states, transition function etc.); implementation: Description of TM using English prose, i.e., how TM moves head, read/writes symbols etc. (without explicitly specifying its states and transition function); high-level: Description of an algorithm using English prose, ignoring implementation details We start with the first two levels and later will switch to the third one, when you feel confident on how to go from one level of algorithm description to another. 37