1. Instructor: Aaron Roth aaroth@cis.upenn.edu
CIS 262 Automata, Computability, and Complexity Spring
Instructor: Aaron Roth
Lecture: March 25, 2019

2. Membership Problem for TMs
Is ATM = { <M, w> | M is a TM and M accepts w } decidable ?
Theorem [Turing, 1936] : ATM is undecidable

3. Undecidability of ATM H Proof by contradiction
Suppose ATM is decidable
Then there exists a halting TM, say, H, such that L(H) = ATM
Given input <M, w>,
if M accepts w, H stops and accepts <M,w>
else H stops and rejects <M,w>
Accept
(M accepts w) H
<M> w
Reject
(otherwise)

4. Undecidability of ATM H H If such a halting TM H for exists,
Accept
(M accepts w)
If such a halting TM H for exists,
Then so does following TM D:
Given input <M>,
if H accepts (<M>,<M>), stop and reject
else stop and accept H
<M> w
Reject
(otherwise) D
Accept
(M accepts <M>)
Reject
<M> H
<M>
<M>
Reject
(otherwise)
Accept

5. Undecidability of ATM H D is a halting TM such that
L(D) = { <M> | M is a TM and M does not accept <M> }
What happens if D is given its own encoding as input ?
Does D accept <D> or not ? D
Accept
(M accepts <M>)
Reject
<M> H
<M>
<M>
Reject
(otherwise)
Accept

6. Undecidability of ATM D is a halting TM such that
L(D) = { <M> | M is a TM and M does not accept <M> }
By definition,
<D> belongs to L(D) if and only if D is a TM and D does not accept <D>
This means:
<D> belongs to L(D) if and only if <D> does not belong to L(D)
Contradiction!
That is, postulated halting TM H cannot exist !

7. Problem Classification
REGULAR
DECIDABLE
HILBERT10
a*b
ADFA
EDFA
PRIMES
EQDFA
ATM
RECOGNIZABLE
ALL LANGUAGES
What are examples of unrecognizable languages ?

8. Countable Sets A brief detour into the concept of “countable sets”
N = Set of natural numbers = { 0, 1, 2, 3, … }
A set S is countable if there is an injective (one-to-one) function f : S  N
Intuitively a set S is countable if we can enumerate the elements of S as first, second, third, and so on, such that each element appears at least once (doesn’t matter if same element appears multiple times)
Set N of natural numbers is countable
Every finite set is countable
Set Z of integers ??
Countable!
Possible enumeration: 0, +1, -1, +2, -2, +3, -3, …

9. Set of pairs of natural numbers = N x N
Countable!
Possible enumeration:
(0,0),
(1,0), (0,1),
(2,0), (1,1), (0,2),
(3,0), (2,1), (1,2), (0,3), …
Same technique shows that:
if S and S’ are countable,
then so is S x S’ 1 2 3 …
(0,0)
(0,1)
(0,2)
(0,3)
(1,0)
(1,1)
(1,2)
(1,3)
(2,0)
(2,1)
(2,2)
(2,3)
(3,0)
(3,1)
(3,2)
(3,3) 4
(4,0)
(4,1)
(4,2)
(4,3)

10. Set of all strings S* Countable!
Enumerate all strings of length 0, then all strings of length 1, then all strings of length of 2, and so on
Enumeration of {0,1}* :
e, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, 100, 101, 110, 111, …

11. Set of all Turing Machines
Is this set countable ?
Yes, each TM M can be encoded as a string <M>
Set of all strings can be enumerated, and hence so can the set of strings that are valid encodings of TMs
Start enumerating strings, but remove those that don’t encode TMs
Thus, we have an enumeration of all Turing machines:
M1, M2, M3, M4, …
By same logic, the set of all programs, say in Java, is countable, and we can enumerate all possible Java programs …

12. Set of Recognizable Languages
Is this set countable ?
Yes !
A language L is recognizable if there exists a TM M such that L(M)=L
Consider an enumeration of all Turing machines:
M1, M2, M3, M4, …
By definition, the following is an enumeration of all recognizable languages
L(M1), L(M2), L(M3), L(M4), …

13. Uncountable Sets 2N = Set of all subsets of N Is 2N countable ?
No, Proof by contradiction:
Suppose 2N is countable
Consider an enumeration S1, S2, S3, , … of 2N
Define the following subset of N:
Sd = { i | i does not belong to the set Si }
By definition, for each i, i belongs to exactly one of Si and Sd,
and thus, Sd != Si
It follows that the set Sd does not appear in this enumeration
Contradiction !

14. Examples of Uncountable Sets
2N = Set of all subsets of N
R = Set of all real numbers
R[0,1) = Set of real numbers in the interval [0, 1)
Aside: there is a one-to-one correspondence between 2N and R[0,1)
Both have same cardinality
Set of all languages over a given alphabet S ??
A language L is a subset of S*
But S* is countable and infinite, has same cardinality as N
One-to-one correspondence between 2N and set of all languages
Uncountable !

15. Existence of Unrecognizable Languages
ALL LANGUAGES
RECOGNIZABLE
Set of all languages is uncountable with cardinality of reals
Set of recognizable languages is countable
Corollary: Most languages are unrecognizable !
Next step: Construct a specific unrecognizable language

16. Sets and Characteristic Vectors
The set S* of all strings is countable
Let’s fix an enumeration of all the strings:
w1, w2, w3, w4, …
A language L is a subset of S*, we can encode it as an infinite vector of 0/1s, called the characteristic vector:
i-th entry is 1 is the string wi is in L, and 0 otherwise
Example:
Set {w1, w3, w4, w6 … } corresponds to vector …
Vector … corresponds to set {w3, w4, w5, … }

17. Diagonalization Construction
The set of all TMs is countable
Let’s fix an enumeration of all the machines:
M1, M2, M3, M4, …
Define infinite matrix C: i-th row is the characteristic vector of L(Mi)
that is, C[i,j] = 1 if TM Mi accepts string wj

18. Diagonalization Construction
C[i,j] = 1 if TM Mi accepts string wj
Consider infinite vector of diagonal entries … w1 w2 w3 w4 … M1 1 M2 M3 M4 M5
Flip all bits: …
Ld = Language corresponding to
this flipped diagonal vector
= {wi | wi is not in L(Mi) }
Is Ld recognizable ?
Can we construct a TM that given an input string w, if w is the i-th string in enumeration of S*, accepts w if the i-th machine Mi does not accept w ?

19. Diagonalization Construction
C[i,j] = 1 if TM Mi accepts string wj w1 w2 w3 w4 … M1 1 M2 M3 M4 M5
Ld = {wi | wi is not in L(Mi) }
Claim: For all Turing machines M, L(M) != Ld
If M is the i-th machine in our enumeration,
then the i-th string wi is in Ld if and only if wi is not accepted by M
Corollary: The language Ld is not recognizable

20. Problem Classification
ALL LANGUAGES
RECOGNIZABLE Ld
DECIDABLE
ATM
How to classify a given problem L as
(1) decidable, (2) recognizable-but-undecidable, or (3) unrecognizable ?
Upper bound techniques:
(a) To show L is decidable, find a halting TM/program for it
(b) To show L is recognizable, find a TM/program that accepts it
How to show L is undecidable, or unrecognizable ?
Use of closure properties
Technique of problem reduction and known results for Ld and ATM

21. Combining Two Recognizers
Theorem: If a language L and its complement ~L are both recognizable, then both most be decidable
Suppose L and ~L to be recognizable
Then there is a TM M for L and another TM M’ for ~L
To get a halting TM T for L, we can run M and M’ in parallel (as in the construction of union of recognizable languages)
Input w is copied on both tapes initially
Repeat {
Execute one transition of M on Tape1;
If M stops and accepts, stop and accept;
Execute one transition of M’ on Tape2;
If M’ stops and accepts, stop and reject }
Input is either in L or in ~L, so one of them has to stop, so T always halts
L(T) = L, so L is decidable, and hence, so is its complement

22. Recap: Known Facts If L is decidable, then L is also recognizable
If L is decidable, then its complement ~L is also decidable
If L1 and L2 are decidable, then union of L1 and L2 is decidable
If L1 and L2 are decidable, then intersection of L1 and L2 is decidable
If L1 and L2 are recognizable, then union of L1 and L2 is recognizable
If L1 and L2 are recognizable, then intersection of L1 and L2 is recognizable
If L and ~L are both recognizable, then both are decidable
ATM = { <M, w> | M is a TM and M accepts w } is undecidable
Ld = {wi | wi is not accepted by TM wi } is unrecognizable

23. Non-membership Problem for TMs
ATM = { <M,w> | M is a TM and accepts w } : recognizable-but-undecidable
Non-membership problem: Given a TM M and w, check if w is not in L(M)
NATM = { <M,w> | M is a TM and does not accept w }
Claim: NATM is unrecognizable
Proof by contradiction:
Suppose NATM is recognizable
Then so is ~ATM
We know that if a language and its negation are both recognizable,
then both must be decidable
So ATM is decidable, contradiction !

24. Halting Problem for TMs
HALTTM = { <M,w> | M is a TM and M halts on w }
Is this problem:
decidable, recognizable-but-undecidable, unrecognizable ??
Claim1: HALTTM is recognizable
Intuitively, no harder than ATM = { <M,w> | M is a TM and M accepts w }
Use construction of universal TM for ATM in this case also

25. Halting Problem for TMs
HALTTM = { <M,w> | M is a TM and M halts on w }
Claim1: HALTTM is recognizable
Consider the following TM M’:
Given input <M,w>,
Check if <M> encodes a TM, if not stop and reject
Execute M step-by-step on input w
If M stops and accepts, then stop and accept <M,w>
If M stops and rejects, then stop and accept <M,w>
L(M’) = { <M,w> | M is a TM and M halts on w }
Note: M’ is not a halting TM, and if M loops forever on input w, so does M’ on input <M,w>

26. Halting Problem for TMs
HALTTM = { <M,w> | M is a TM and M halts on w }
Known fact: ATM = { <M,w> | M accepts w } is undecidable
And HALTTM problem does not seem any easier
Goal: Prove: Undecidability of ATM implies undecidability of HALTTM
Language / Problem reduction:
Show that ATM reduces to HALTTM
That is, show how to construct a TM for ATM
using a TM for HALTTM as a subroutine

27. Halting Problem for TMs
HALTTM = { <M,w> | M is a TM and M halts on w }
ATM = { <M,w> | M is a TM and M accepts w }
Consider a halting TM H that solves HALTTM
Goal: Build a halting TM U for ATM
Given an input <M, w>, where <M> indeed encodes a TM,
U needs to decide whether or not M accepts w
If U were to just execute M on w, it may not halt, if M doesn’t halt on w
If U could check in advance whether M halts on w or not, this would not be a problem, and this desired test can be implemented by calling H !

28. Reducing ATM to HALTTM
HALTTM = { <M,w> | M is a TM and M halts on w }
ATM = { <M,w> | M is a TM and M accepts w }
Suppose there is a halting TM H that solves HALTTM
Define the following TM U :
Given input <M, w>:
If <M> does not encode a TM, then reject
Execute H on input <M, w>
If H rejects, then reject
Else execute M on input w
If M accepts, then accept, else reject
Claim: If H is a halting TM for HALTTM, U is a halting TM for ATM
U executes M only when
it is assured of halting

29. Proof by Problem Reduction
HALTTM = { <M,w> | M is a TM and M halts on w }
ATM = { <M,w> | M is a TM and M accepts w }
Claim: If H is a halting TM for HALTTM, U is a halting TM for ATM
Follows that if HALTTM is decidable, ATM is also decidable
Conversely, if ATM is undecidable, HALTTM is also undecidable
But we have proved that ATM is undecidable
Conclusion: HALTTM is undecidable