Instructor: Aaron Roth aaroth@cis.upenn.edu CIS 262 Automata, Computability, and Complexity Spring 2019 http://www.seas.upenn.edu/~cse262/ Instructor: Aaron Roth aaroth@cis.upenn.edu Lecture: March 20, 2019

Closure Properties of Recognizable Languages A language L is recognizable if there exists a TM M such that L(M)=L Note: it is possible that M loops on some inputs Are recognizable languages closed under union? Yes means: if two languages are recognizable, then their union is guaranteed to be recognizable Consider two recognizable languages L1 and L2 There exist TMs M1 and M2 such that L(M1)= L1 and L(M2)= L2 Goal: show that there exists a TM M such that M accepts those inputs w exactly when at least one of M1 and M2 do

Recognizable Languages and Union Consider two recognizable languages L1 and L2 There exist TMs M1 and M2 such that L(M1)= L1 and L(M2)= L2 Consider the following TM M: Given an input w, Execute M1 on w; If M1 stops and accepts then accept Else { Execute M2 on w; If M2 stops and accepts, accept } Is it the case that L(M) = Union of L1 and L2 ? May not terminate, so when w is not in L1 but is in L2, M may not accept w as required

Fixing the construction for Union M should stop and accept if either one of M1 and M2 does Key idea: need to run both machines without committing to one Basically, we want to run M1 and M2 in parallel, but our computational model does not explicitly support parallelism, so simulate it by alternating steps of the two on two tapes: Input w is copied on both tapes initially Repeat { Execute one transition of M1 on Tape1; If M1 stops and accepts, stop and accept; Execute one transition of M2 on Tape2; If M2 stops and accepts, stop and accept } M accepts w if and only if either M1 accepts w or M2 accepts w

Recognizable Languages and Intersection Goal: Prove that recognizable languages are closed under intersection Given TMs M1 and M2, consider the following TM M: Given an input w, Execute M1 on w; If M1 stops and accepts then { Execute M2 on w; If M2 stops and accepts, accept else reject } Else reject Can we claim that L(M) = Intersection of L(M1) and L(M2) May not terminate, when w is not in L1 , but in this case, M has no obligation

Recognizable Languages and Complement If L is a recognizable language, what can we say about ~L ? Take TM M for L, just switch the roles of accepting and rejecting states to get the TM ~M, is it the case that L(M) = ~ L(~M) ? On a given input w, 1. if M stops and accepts w, then ~M stops and rejects w 2. if M stops and rejects w, then ~M stops and accepts w 3. if M loops forever on w, so does ~M Thus, unless we know that M is halting, cannot conclude L(M)=~L(~M) In fact, the class of recognizable language is not closed under complement !

Closure under Concatenation A string w belongs to L1 and L2 if w can be split into two parts w = u.v such that u is in L1 and v is in L2 Are recognizable languages closed under concatenation ? Given TMs M1 and M2 for L1 and L2, respectively, can we systematically build a TM M for L1 . L2 ?

Recognizable Languages and Concatenation The desired TM M needs to try all ways of splitting its input, but for each split execute the two machines only for bounded number of steps: Given an input w with length(w)=k, n = 1; repeat { for i = 0 to k { Execute M1 for n steps on prefix of w with i symbols; If M1 accepts then { Execute M2 for n steps on suffix of w with k-i symbols; If M2 accepts, stop and accept } }; n = n+1; }

Recognizable Languages and Concatenation Correctness of construction: M stops and accepts only when it has found a way to split input w into two parts w = u. v such that M1 accepts u and M2 accepts v, and thus, belongs to L1 . L2 Inner “for” loop always terminates since individual machines are executed for n steps If w is in L1 . L2, then there exists a split w = u.v such that M1 accepts u and M2 accepts v Suppose accepting execution of M1 on u consists of n1 steps and accepting execution of M2 on v consists of n2 steps Then, when n equals max(n1, n2), M will stop and accept

Problems about Machines Membership problem for DFAs: Given a DFA M and string w, does M accept w ? First, we need to figure out a way to encode DFAs as strings: <M> : encoding of DFA M as a string Recall a DFA M consists of set Q of states, set S of input alphabet, initial state q0, final states F, and transition function d d can be encoded as a list of triples (old-state, symbol, new-state) For our purpose, what matters is that there is a way to encode the description of M, and not the exact details of how it’s done <M,w>: Input is a pair, first encodes DFA M, and second input for M

Problems about Machines as Languages Membership problem for DFAs: Given a DFA M and string w, does M accept w ? ADFA = { <M, w> | M is a DFA and M accepts w } What does “M is a DFA” mean?? For a DFA M, <M> denotes the string that encodes its description, but not every string is an encoding of a DFA So, a machine for ADFA expects its input to be a valid encoding of a DFA, it first checks that, and if so, then needs to check if the DFA it encodes has the desired property, that is, accepts the string w, encoded as the second component of the input

Membership Problem for DFAs ADFA = { <M, w> | M is a DFA and M accepts w } This language is decidable The halting TM/program P for ADFA : First check if the input is well-formed and indeed encodes a DFA M If not, stop and reject If so, execute DFA M on input w and check M ends up in its final state If so, stop and accept, else reject Note that the machine P itself is not a DFA, but a Turing machine The language ADFA is decidable, but not regular

Emptiness Problem for DFAs EDFA = { <M> | M is a DFA and L(M) is empty } Is this language decidable ? The halting TM/program P for EDFA : First check if the input is well-formed and indeed encodes a DFA M If not, stop and reject If so, compute set of states of M reachable from its initial state If this set contains some final state of M, Then stop and reject (M does accept some strings in this case) Else stop and accept (language of M is empty in this case)

Equivalence Problem for DFAs EQDFA = { <M, M’> | M and M’ are DFAs and L(M) equals L(M’) } Recall the algorithm for checking equivalence of DFAs: L(M) = L(M’) iff both 1. Intersection of L(M) and ~L(M’) is empty 2. Intersection of ~L(M) and L(M’) is empty To check 1, take product of M and negation of M’ (obtained by flipping final/non-final states), and check if its language is empty Condition 2 is checked similarly Conclusion: EQDFA is decidable

Problems about NFAs and Regular Expressions EQNFA = { <M, M’> | M and M’ are NFAs and L(M) equals L(M’) } Decidable ! First check that input encodes NFAs, then convert them to DFAs using the subset construction, and then check equivalence of resulting DFAs Similarly, many problems about NFAs, regular expressions are decidable

Decision problems about DFAs are decidable REGULAR ADFA EDFA HILBERT10 a*b EQDFA PRIMES EQNFA RECOGNIZABLE ALL LANGUAGES

Semantic Problems about TMs ATM = { <M, w> | M is a TM and M accepts input w } ETM = { <M> | M is a TM and L(M) is empty } EQTM = { <M, M’> | M and M’ are TMs, and L(M)=L(M’) } A program/TM to solve such a question needs to analyze what the input TM computes Coming up: all these questions are undecidable and so are similar questions about programs written in modern programming languages (e.g. does a program terminate?)

Membership Problem for Turing Machines ATM = { <M, w> | M is a TM and M accepts w } What does the machine U to solve this problem need to do ? First step: check if input indeed encodes a TM, if not, stop and reject In sequel, let’s just assume that input encodes a valid TM Given the description <M> of a Turing machine as input and a string w, the desired machine U needs to “simulate” execution of M on w Does such a program/TM U exist ?

Universal Turing Machine U Tape-1 Encoding of the input TM M Tape-2 Tape of M (initially contains the input w) Tape-3 State of M (initially q0) repeat { Find transition on Tape1 that matches state on Tape3 and symbol read by Head2 on Tape2; Update state on Tape3, symbol at Head2 on tape2, and move Head2; If state of M on Tape3 equals qa, stop and accept; If state of M on Tape3 equals qr, stop and reject; }

Universal Turing Machine U Given <M> and w as inputs, U maintains state and tape content of M on its tapes and updates them step-by-step applying transitions of M In modern programming language terminology, U is an interpreter for Turing machines, and can itself be implemented as a Turing machine What is L(U) ? 1. If M accepts w, then U accepts <M,w> 2. If M rejects w, then U rejects <M,w> 4. If M does not halt on w, then U does not halt on <M,w> Claim: L(U) = ATM = { <M, w> | M is a TM and M accepts w } Hence, ATM is recognizable (U is not a halting TM, so can’t say decidable)

Membership Problem for TMs Is ATM = { <M, w> | M is a TM and M accepts w } decidable ? The fact that our universal TM U is not a halting TM does not establish undecidability of ATM , but … Theorem [Turing, 1936] : ATM is undecidable In its original form: Halting problem for TMs is undecidable HALTTM = { <M, w> | M is a TM and M halts on w }

Undecidability of ATM H Proof by contradiction Suppose ATM is decidable Then there exists a halting TM, say, H, such that L(H) = ATM Given input <M, w>, if M accepts w, H stops and accepts <M,w> else H stops and rejects <M,w> Accept (M accepts w) H <M> w Reject (otherwise)

Undecidability of ATM H H If such a halting TM H for exists, Accept (M accepts w) If such a halting TM H for exists, Then so does following TM D: Given input <M>, if H accepts (<M>,<M>), stop and reject else stop and accept H <M> w Reject (otherwise) D Accept (M accepts <M>) Reject <M> H <M> <M> Reject (otherwise) Accept

Undecidability of ATM H D is a halting TM such that L(D) = { <M> | M is a TM and M does not accept <M> } What happens if D is given its own encoding as input ? Does D accept <D> or not ? D Accept (M accepts <M>) Reject <M> H <M> <M> Reject (otherwise) Accept

Undecidability of ATM D is a halting TM such that L(D) = { <M> | M is a TM and M does not accept <M> } By definition, <D> belongs to L(D) if and only if D is a TM and D does not accept <D> This means: <D> belongs to L(D) if and only if <D> does not belong to L(D) Contradiction! That is, postulated halting TM H cannot exist !

Undecidability of ATM H In reality, any attempt to implement H will not give a halting TM: If <M> goes into a loop on the input <M>, H also goes into a loop, and so does D Thus, even when M does not accept <M>, D is not able to accept <M>, and thus L(D) is not { <M> | M is a TM that does not accept <M> } D Accept (M accepts <M>) Reject <M> H <M> <M> Reject (otherwise) Accept

Undecidability of Halting Problem Turing’s Theorem [1936]: Halting problem for Turing machines is undecidable Establishes basic limits on what computers can analyze about other programs/machines Intuitively, the proof says that if a machine model is universal (that is, can simulate other machines), then its own behavior cannot be analyzed by a machine (a program cannot tell correctly if it itself is making an error, such as going into an infinite loop !)

Problem Classification REGULAR DECIDABLE HILBERT10 a*b ADFA EDFA PRIMES EQDFA ATM RECOGNIZABLE ALL LANGUAGES What are examples of unrecognizable languages ?