1. Instructor: Aaron Roth aaroth@cis.upenn.edu
CIS 262 Automata, Computability, and Complexity Spring
Instructor: Aaron Roth
Lecture: March 20, 2019

2. Closure Properties of Recognizable Languages
A language L is recognizable if there exists a TM M such that L(M)=L
Note: it is possible that M loops on some inputs
Are recognizable languages closed under union?
Yes means: if two languages are recognizable, then their union is guaranteed to be recognizable
Consider two recognizable languages L1 and L2
There exist TMs M1 and M2 such that L(M1)= L1 and L(M2)= L2
Goal: show that there exists a TM M such that M accepts those inputs w exactly when at least one of M1 and M2 do

3. Recognizable Languages and Union
Consider two recognizable languages L1 and L2
There exist TMs M1 and M2 such that L(M1)= L1 and L(M2)= L2
Consider the following TM M:
Given an input w,
Execute M1 on w;
If M1 stops and accepts then accept
Else {
Execute M2 on w;
If M2 stops and accepts, accept }
Is it the case that L(M) = Union of L1 and L2 ?
May not terminate, so when w is not in L1 but is in L2, M may not accept w as required

4. Fixing the construction for Union
M should stop and accept if either one of M1 and M2 does
Key idea: need to run both machines without committing to one
Basically, we want to run M1 and M2 in parallel, but our computational model does not explicitly support parallelism, so simulate it by alternating steps of the two on two tapes:
Input w is copied on both tapes initially
Repeat {
Execute one transition of M1 on Tape1;
If M1 stops and accepts, stop and accept;
Execute one transition of M2 on Tape2;
If M2 stops and accepts, stop and accept }
M accepts w if and only if either M1 accepts w or M2 accepts w

5. Recognizable Languages and Intersection
Goal: Prove that recognizable languages are closed under intersection
Given TMs M1 and M2, consider the following TM M:
Given an input w,
Execute M1 on w;
If M1 stops and accepts then {
Execute M2 on w;
If M2 stops and accepts, accept else reject }
Else reject
Can we claim that L(M) = Intersection of L(M1) and L(M2)
May not terminate, when w is not in L1 , but in this case, M has no obligation

6. Recognizable Languages and Complement
If L is a recognizable language, what can we say about ~L ?
Take TM M for L, just switch the roles of accepting and rejecting states to get the TM ~M, is it the case that L(M) = ~ L(~M) ?
On a given input w,
1. if M stops and accepts w, then ~M stops and rejects w
2. if M stops and rejects w, then ~M stops and accepts w
3. if M loops forever on w, so does ~M
Thus, unless we know that M is halting, cannot conclude L(M)=~L(~M)
In fact, the class of recognizable language is not closed under complement !

7. Closure under Concatenation
A string w belongs to L1 and L2 if w can be split into two parts w = u.v such that u is in L1 and v is in L2
Are recognizable languages closed under concatenation ?
Given TMs M1 and M2 for L1 and L2, respectively, can we systematically build a TM M for L1 . L2 ?

8. Recognizable Languages and Concatenation
The desired TM M needs to try all ways of splitting its input, but for each split execute the two machines only for bounded number of steps:
Given an input w with length(w)=k,
n = 1;
repeat {
for i = 0 to k {
Execute M1 for n steps on prefix of w with i symbols;
If M1 accepts then {
Execute M2 for n steps on suffix of w with k-i symbols;
If M2 accepts, stop and accept } };
n = n+1; }

9. Recognizable Languages and Concatenation
Correctness of construction:
M stops and accepts only when it has found a way to split input w into two parts w = u. v such that M1 accepts u and M2 accepts v, and thus, belongs to L1 . L2
Inner “for” loop always terminates since individual machines are executed for n steps
If w is in L1 . L2, then there exists a split w = u.v such that M1 accepts u and M2 accepts v
Suppose accepting execution of M1 on u consists of n1 steps and accepting execution of M2 on v consists of n2 steps
Then, when n equals max(n1, n2), M will stop and accept

10. Problems about Machines
Membership problem for DFAs:
Given a DFA M and string w, does M accept w ?
First, we need to figure out a way to encode DFAs as strings:
<M> : encoding of DFA M as a string
Recall a DFA M consists of set Q of states, set S of input alphabet, initial state q0, final states F, and transition function d
d can be encoded as a list of triples (old-state, symbol, new-state)
For our purpose, what matters is that there is a way to encode the description of M, and not the exact details of how it’s done
<M,w>: Input is a pair, first encodes DFA M, and second input for M

11. Problems about Machines as Languages
Membership problem for DFAs:
Given a DFA M and string w, does M accept w ?
ADFA = { <M, w> | M is a DFA and M accepts w }
What does “M is a DFA” mean??
For a DFA M, <M> denotes the string that encodes its description, but not every string is an encoding of a DFA
So, a machine for ADFA expects its input to be a valid encoding of a DFA, it first checks that, and if so, then needs to check if the DFA it encodes has the desired property, that is, accepts the string w, encoded as the second component of the input

12. Membership Problem for DFAs
ADFA = { <M, w> | M is a DFA and M accepts w }
This language is decidable
The halting TM/program P for ADFA :
First check if the input is well-formed and indeed encodes a DFA M
If not, stop and reject
If so, execute DFA M on input w and check M ends up in its final state
If so, stop and accept, else reject
Note that the machine P itself is not a DFA, but a Turing machine
The language ADFA is decidable, but not regular

13. Emptiness Problem for DFAs
EDFA = { <M> | M is a DFA and L(M) is empty }
Is this language decidable ?
The halting TM/program P for EDFA :
First check if the input is well-formed and indeed encodes a DFA M
If not, stop and reject
If so, compute set of states of M reachable from its initial state
If this set contains some final state of M,
Then stop and reject (M does accept some strings in this case)
Else stop and accept (language of M is empty in this case)

14. Equivalence Problem for DFAs
EQDFA = { <M, M’> | M and M’ are DFAs and L(M) equals L(M’) }
Recall the algorithm for checking equivalence of DFAs:
L(M) = L(M’) iff both
1. Intersection of L(M) and ~L(M’) is empty
2. Intersection of ~L(M) and L(M’) is empty
To check 1, take product of M and negation of M’ (obtained by flipping final/non-final states), and check if its language is empty
Condition 2 is checked similarly
Conclusion: EQDFA is decidable

15. Problems about NFAs and Regular Expressions
EQNFA = { <M, M’> | M and M’ are NFAs and L(M) equals L(M’) }
Decidable !
First check that input encodes NFAs, then convert them to DFAs using the subset construction, and then check equivalence of resulting DFAs
Similarly, many problems about NFAs, regular expressions are decidable

16. Decision problems about DFAs are decidable
REGULAR
ADFA
EDFA
HILBERT10
a*b
EQDFA
PRIMES
EQNFA
RECOGNIZABLE
ALL LANGUAGES

17. Semantic Problems about TMs
ATM = { <M, w> | M is a TM and M accepts input w }
ETM = { <M> | M is a TM and L(M) is empty }
EQTM = { <M, M’> | M and M’ are TMs, and L(M)=L(M’) }
A program/TM to solve such a question needs to analyze what the input TM computes
Coming up: all these questions are undecidable
and so are similar questions about programs written in modern programming languages (e.g. does a program terminate?)

18. Membership Problem for Turing Machines
ATM = { <M, w> | M is a TM and M accepts w }
What does the machine U to solve this problem need to do ?
First step: check if input indeed encodes a TM, if not, stop and reject
In sequel, let’s just assume that input encodes a valid TM
Given the description <M> of a Turing machine as input and a string w,
the desired machine U needs to “simulate” execution of M on w
Does such a program/TM U exist ?

19. Universal Turing Machine U
Tape-1
Encoding of the input TM M
Tape-2
Tape of M (initially contains the input w)
Tape-3
State of M (initially q0)
repeat {
Find transition on Tape1 that matches state on Tape3 and
symbol read by Head2 on Tape2;
Update state on Tape3, symbol at Head2 on tape2, and move Head2;
If state of M on Tape3 equals qa, stop and accept;
If state of M on Tape3 equals qr, stop and reject; }

20. Universal Turing Machine U
Given <M> and w as inputs,
U maintains state and tape content of M on its tapes
and updates them step-by-step applying transitions of M
In modern programming language terminology, U is an interpreter for Turing machines, and can itself be implemented as a Turing machine
What is L(U) ?
1. If M accepts w, then U accepts <M,w>
2. If M rejects w, then U rejects <M,w>
4. If M does not halt on w, then U does not halt on <M,w>
Claim: L(U) = ATM = { <M, w> | M is a TM and M accepts w }
Hence, ATM is recognizable (U is not a halting TM, so can’t say decidable)

21. Membership Problem for TMs
Is ATM = { <M, w> | M is a TM and M accepts w } decidable ?
The fact that our universal TM U is not a halting TM does not establish undecidability of ATM , but …
Theorem [Turing, 1936] : ATM is undecidable
In its original form: Halting problem for TMs is undecidable
HALTTM = { <M, w> | M is a TM and M halts on w }

22. Undecidability of ATM H Proof by contradiction
Suppose ATM is decidable
Then there exists a halting TM, say, H, such that L(H) = ATM
Given input <M, w>,
if M accepts w, H stops and accepts <M,w>
else H stops and rejects <M,w>
Accept
(M accepts w) H
<M> w
Reject
(otherwise)

23. Undecidability of ATM H H If such a halting TM H for exists,
Accept
(M accepts w)
If such a halting TM H for exists,
Then so does following TM D:
Given input <M>,
if H accepts (<M>,<M>), stop and reject
else stop and accept H
<M> w
Reject
(otherwise) D
Accept
(M accepts <M>)
Reject
<M> H
<M>
<M>
Reject
(otherwise)
Accept

24. Undecidability of ATM H D is a halting TM such that
L(D) = { <M> | M is a TM and M does not accept <M> }
What happens if D is given its own encoding as input ?
Does D accept <D> or not ? D
Accept
(M accepts <M>)
Reject
<M> H
<M>
<M>
Reject
(otherwise)
Accept

25. Undecidability of ATM D is a halting TM such that
L(D) = { <M> | M is a TM and M does not accept <M> }
By definition,
<D> belongs to L(D) if and only if D is a TM and D does not accept <D>
This means:
<D> belongs to L(D) if and only if <D> does not belong to L(D)
Contradiction!
That is, postulated halting TM H cannot exist !

26. Undecidability of ATM H
In reality, any attempt to implement H will not give a halting TM:
If <M> goes into a loop on the input <M>,
H also goes into a loop, and so does D
Thus, even when M does not accept <M>, D is not able to accept <M>, and thus L(D) is not { <M> | M is a TM that does not accept <M> } D
Accept
(M accepts <M>)
Reject
<M> H
<M>
<M>
Reject
(otherwise)
Accept

27. Undecidability of Halting Problem
Turing’s Theorem [1936]: Halting problem for Turing machines is undecidable
Establishes basic limits on what computers can analyze about other programs/machines
Intuitively, the proof says that if a machine model is universal (that is, can simulate other machines), then its own behavior cannot be analyzed by a machine (a program cannot tell correctly if it itself is making an error, such as going into an infinite loop !)

28. Problem Classification
REGULAR
DECIDABLE
HILBERT10
a*b
ADFA
EDFA
PRIMES
EQDFA
ATM
RECOGNIZABLE
ALL LANGUAGES
What are examples of unrecognizable languages ?