Instructor: Aaron Roth aaroth@cis.upenn.edu CIS 262 Automata, Computability, and Complexity Spring 2019 http://www.seas.upenn.edu/~cse262/ Instructor: Aaron Roth aaroth@cis.upenn.edu Lecture: March 25, 2019

Membership Problem for TMs Is ATM = { <M, w> | M is a TM and M accepts w } decidable ? Theorem [Turing, 1936] : ATM is undecidable

Undecidability of ATM H Proof by contradiction Suppose ATM is decidable Then there exists a halting TM, say, H, such that L(H) = ATM Given input <M, w>, if M accepts w, H stops and accepts <M,w> else H stops and rejects <M,w> Accept (M accepts w) H <M> w Reject (otherwise)

Undecidability of ATM H H If such a halting TM H for exists, Accept (M accepts w) If such a halting TM H for exists, Then so does following TM D: Given input <M>, if H accepts (<M>,<M>), stop and reject else stop and accept H <M> w Reject (otherwise) D Accept (M accepts <M>) Reject <M> H <M> <M> Reject (otherwise) Accept

Undecidability of ATM H D is a halting TM such that L(D) = { <M> | M is a TM and M does not accept <M> } What happens if D is given its own encoding as input ? Does D accept <D> or not ? D Accept (M accepts <M>) Reject <M> H <M> <M> Reject (otherwise) Accept

Undecidability of ATM D is a halting TM such that L(D) = { <M> | M is a TM and M does not accept <M> } By definition, <D> belongs to L(D) if and only if D is a TM and D does not accept <D> This means: <D> belongs to L(D) if and only if <D> does not belong to L(D) Contradiction! That is, postulated halting TM H cannot exist !

Problem Classification REGULAR DECIDABLE HILBERT10 a*b ADFA EDFA PRIMES EQDFA ATM RECOGNIZABLE ALL LANGUAGES What are examples of unrecognizable languages ?

Countable Sets A brief detour into the concept of “countable sets” N = Set of natural numbers = { 0, 1, 2, 3, … } A set S is countable if there is an injective (one-to-one) function f : S  N Intuitively a set S is countable if we can enumerate the elements of S as first, second, third, and so on, such that each element appears at least once (doesn’t matter if same element appears multiple times) Set N of natural numbers is countable Every finite set is countable Set Z of integers ?? Countable! Possible enumeration: 0, +1, -1, +2, -2, +3, -3, …

Set of pairs of natural numbers = N x N Countable! Possible enumeration: (0,0), (1,0), (0,1), (2,0), (1,1), (0,2), (3,0), (2,1), (1,2), (0,3), … Same technique shows that: if S and S’ are countable, then so is S x S’ 1 2 3 … (0,0) (0,1) (0,2) (0,3) (1,0) (1,1) (1,2) (1,3) (2,0) (2,1) (2,2) (2,3) (3,0) (3,1) (3,2) (3,3) 4 (4,0) (4,1) (4,2) (4,3)

Set of all strings S* Countable! Enumerate all strings of length 0, then all strings of length 1, then all strings of length of 2, and so on Enumeration of {0,1}* : e, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, 100, 101, 110, 111, …

Set of all Turing Machines Is this set countable ? Yes, each TM M can be encoded as a string <M> Set of all strings can be enumerated, and hence so can the set of strings that are valid encodings of TMs Start enumerating strings, but remove those that don’t encode TMs Thus, we have an enumeration of all Turing machines: M1, M2, M3, M4, … By same logic, the set of all programs, say in Java, is countable, and we can enumerate all possible Java programs …

Set of Recognizable Languages Is this set countable ? Yes ! A language L is recognizable if there exists a TM M such that L(M)=L Consider an enumeration of all Turing machines: M1, M2, M3, M4, … By definition, the following is an enumeration of all recognizable languages L(M1), L(M2), L(M3), L(M4), …

Uncountable Sets 2N = Set of all subsets of N Is 2N countable ? No, Proof by contradiction: Suppose 2N is countable Consider an enumeration S1, S2, S3, , … of 2N Define the following subset of N: Sd = { i | i does not belong to the set Si } By definition, for each i, i belongs to exactly one of Si and Sd, and thus, Sd != Si It follows that the set Sd does not appear in this enumeration Contradiction !

Examples of Uncountable Sets 2N = Set of all subsets of N R = Set of all real numbers R[0,1) = Set of real numbers in the interval [0, 1) Aside: there is a one-to-one correspondence between 2N and R[0,1) Both have same cardinality Set of all languages over a given alphabet S ?? A language L is a subset of S* But S* is countable and infinite, has same cardinality as N One-to-one correspondence between 2N and set of all languages Uncountable !

Existence of Unrecognizable Languages ALL LANGUAGES RECOGNIZABLE Set of all languages is uncountable with cardinality of reals Set of recognizable languages is countable Corollary: Most languages are unrecognizable ! Next step: Construct a specific unrecognizable language

Sets and Characteristic Vectors The set S* of all strings is countable Let’s fix an enumeration of all the strings: w1, w2, w3, w4, … A language L is a subset of S*, we can encode it as an infinite vector of 0/1s, called the characteristic vector: i-th entry is 1 is the string wi is in L, and 0 otherwise Example: Set {w1, w3, w4, w6 … } corresponds to vector 1 0 1 1 0 1 … Vector 0 0 1 1 1 0 … corresponds to set {w3, w4, w5, … }

Diagonalization Construction The set of all TMs is countable Let’s fix an enumeration of all the machines: M1, M2, M3, M4, … Define infinite matrix C: i-th row is the characteristic vector of L(Mi) that is, C[i,j] = 1 if TM Mi accepts string wj

Diagonalization Construction C[i,j] = 1 if TM Mi accepts string wj Consider infinite vector of diagonal entries 0 0 1 0 1 … w1 w2 w3 w4 … M1 1 M2 M3 M4 M5 Flip all bits: 1 1 0 1 0 … Ld = Language corresponding to this flipped diagonal vector = {wi | wi is not in L(Mi) } Is Ld recognizable ? Can we construct a TM that given an input string w, if w is the i-th string in enumeration of S*, accepts w if the i-th machine Mi does not accept w ?

Diagonalization Construction C[i,j] = 1 if TM Mi accepts string wj w1 w2 w3 w4 … M1 1 M2 M3 M4 M5 Ld = {wi | wi is not in L(Mi) } Claim: For all Turing machines M, L(M) != Ld If M is the i-th machine in our enumeration, then the i-th string wi is in Ld if and only if wi is not accepted by M Corollary: The language Ld is not recognizable

Problem Classification ALL LANGUAGES RECOGNIZABLE Ld DECIDABLE ATM How to classify a given problem L as (1) decidable, (2) recognizable-but-undecidable, or (3) unrecognizable ? Upper bound techniques: (a) To show L is decidable, find a halting TM/program for it (b) To show L is recognizable, find a TM/program that accepts it How to show L is undecidable, or unrecognizable ? Use of closure properties Technique of problem reduction and known results for Ld and ATM

Combining Two Recognizers Theorem: If a language L and its complement ~L are both recognizable, then both most be decidable Suppose L and ~L to be recognizable Then there is a TM M for L and another TM M’ for ~L To get a halting TM T for L, we can run M and M’ in parallel (as in the construction of union of recognizable languages) Input w is copied on both tapes initially Repeat { Execute one transition of M on Tape1; If M stops and accepts, stop and accept; Execute one transition of M’ on Tape2; If M’ stops and accepts, stop and reject } Input is either in L or in ~L, so one of them has to stop, so T always halts L(T) = L, so L is decidable, and hence, so is its complement

Recap: Known Facts If L is decidable, then L is also recognizable If L is decidable, then its complement ~L is also decidable If L1 and L2 are decidable, then union of L1 and L2 is decidable If L1 and L2 are decidable, then intersection of L1 and L2 is decidable If L1 and L2 are recognizable, then union of L1 and L2 is recognizable If L1 and L2 are recognizable, then intersection of L1 and L2 is recognizable If L and ~L are both recognizable, then both are decidable ATM = { <M, w> | M is a TM and M accepts w } is undecidable Ld = {wi | wi is not accepted by TM wi } is unrecognizable

Non-membership Problem for TMs ATM = { <M,w> | M is a TM and accepts w } : recognizable-but-undecidable Non-membership problem: Given a TM M and w, check if w is not in L(M) NATM = { <M,w> | M is a TM and does not accept w } Claim: NATM is unrecognizable Proof by contradiction: Suppose NATM is recognizable Then so is ~ATM We know that if a language and its negation are both recognizable, then both must be decidable So ATM is decidable, contradiction !

Halting Problem for TMs HALTTM = { <M,w> | M is a TM and M halts on w } Is this problem: decidable, recognizable-but-undecidable, unrecognizable ?? Claim1: HALTTM is recognizable Intuitively, no harder than ATM = { <M,w> | M is a TM and M accepts w } Use construction of universal TM for ATM in this case also

Halting Problem for TMs HALTTM = { <M,w> | M is a TM and M halts on w } Claim1: HALTTM is recognizable Consider the following TM M’: Given input <M,w>, Check if <M> encodes a TM, if not stop and reject Execute M step-by-step on input w If M stops and accepts, then stop and accept <M,w> If M stops and rejects, then stop and accept <M,w> L(M’) = { <M,w> | M is a TM and M halts on w } Note: M’ is not a halting TM, and if M loops forever on input w, so does M’ on input <M,w>

Halting Problem for TMs HALTTM = { <M,w> | M is a TM and M halts on w } Known fact: ATM = { <M,w> | M accepts w } is undecidable And HALTTM problem does not seem any easier Goal: Prove: Undecidability of ATM implies undecidability of HALTTM Language / Problem reduction: Show that ATM reduces to HALTTM That is, show how to construct a TM for ATM using a TM for HALTTM as a subroutine

Halting Problem for TMs HALTTM = { <M,w> | M is a TM and M halts on w } ATM = { <M,w> | M is a TM and M accepts w } Consider a halting TM H that solves HALTTM Goal: Build a halting TM U for ATM Given an input <M, w>, where <M> indeed encodes a TM, U needs to decide whether or not M accepts w If U were to just execute M on w, it may not halt, if M doesn’t halt on w If U could check in advance whether M halts on w or not, this would not be a problem, and this desired test can be implemented by calling H !

Reducing ATM to HALTTM HALTTM = { <M,w> | M is a TM and M halts on w } ATM = { <M,w> | M is a TM and M accepts w } Suppose there is a halting TM H that solves HALTTM Define the following TM U : Given input <M, w>: If <M> does not encode a TM, then reject Execute H on input <M, w> If H rejects, then reject Else execute M on input w If M accepts, then accept, else reject Claim: If H is a halting TM for HALTTM, U is a halting TM for ATM U executes M only when it is assured of halting

Proof by Problem Reduction HALTTM = { <M,w> | M is a TM and M halts on w } ATM = { <M,w> | M is a TM and M accepts w } Claim: If H is a halting TM for HALTTM, U is a halting TM for ATM Follows that if HALTTM is decidable, ATM is also decidable Conversely, if ATM is undecidable, HALTTM is also undecidable But we have proved that ATM is undecidable Conclusion: HALTTM is undecidable