1. 1 Incomputable Languages Zeph Grunschlag

2. 2 Announcements HW Due Tuesday

3. 3 Agenda Incomputable Languages Existence Proof Explicit undecidable language: A TM Unrecognizable Language Neither recognizable nor corecognizable Example of a “real world” unsolvable problem

4. 4 Incomputable Languages Are there questions of a computational nature which cannot be answered methodically? We saw last time that the computability viewpoint is to encode algorithmic problems as formal languages and then ask: Are there undecidable languages? Are there languages which are not even recognizable? Can we construct a specific language that’s undecidable? Are there “useful” undecidable languages?

5. 5 Incomputable Languages Existential Proof We start first with an indirect proof that an undecidable language exists. In fact, we can show that a language exists that is neither recognizable nor corecognizable: I.e., there is no TM which halts on all positive instances, and there is no TM which halts on all negative instances so we have no way of necessarily telling when an arbitrary string is or isn’t in the language. The proof is “existential” in that a counting argument is used to show that the language must exist, but we don’t know a priori what the language is.

6. 6 Incomputable Languages Existence Proof The idea is simple: We show that there are many more languages than Turing machines. Every TM can be encoded in binary by some string. Consequently the cardinality 1 of the set of TM’s is no greater than that of {0,1}*. Recall the notion of countable : A set S is countable if there is a 1-to-1 function f : S  N from S to the natural numbers. Equivalently, an infinite set is countable if there is a bijection N  S. THM: The set of bit strings {0,1}* is countable.

7. 7 Countability of {0,1}* Proof. Intuitively we can strike up the 1-to-1 correspondence f : N  {0,1}* by listing the strings in short-lex order:  0 1 00 01 10 11 000 001 010 011 100… 1 2 3 4 5 6 7 8 9 10 11 12… Can even give recursive definition for f : Can then use induction to prove that f is 1-to-1 and onto.

8. 8 Countability of {TM-Languages} Consequently, as every TM is described by a bit string, there can only be as many TM’s as bit strings. In particular, the following sets are countable: L 1 = {recognizable languages} L 2 = {corecognizable languages} DEF: A language L over the alphabet  is corecognizable if  *-L is recognizable.

9. 9 Countability of {TM-Languages} Therefore, L 1  L 2 = {languages which are accepted or rejected by a TM} is a union of countable sets so is countable itself. However, next we will see that the power set P ({0,1}*) –the set of binary languages!– is uncountable. As a corollary, there must be strictly more binary languages than languages in L 1  L 2. Therefore: THM: A language of bit strings exists which is neither recognizable nor corecognizable. So let’s prove that P ({0,1}*) is uncountable:

10. 10 Uncountability of P (  *) Notice that any subset T in P (  *) can be viewed as a function  *  {0,1}. This is done by making the function take the value 1 for elements in the set, and the value 0 for elements not in the set. EG: Consider the function f :  *  {0,1} defined by the following table: Q: What language does f represent? x  01000110110000010100111001011110000… f (x)111100110100111…

11. 11 Uncountability of P (  *) A: f represents pal. Suppose that P (  *) were countable. Thus it would be possible to find a bijection from N to P (  *) and hence list all binary languages as a sequence L 1, L 2, L 3, L 4, L 5, L 6, L 7, … supposedly containing every language of bit strings. The lists creates a table whose rows describe the associated function for each subset: x  01000110110000010100111001011110000… f (x)111100110100111…

12. 12 Uncountability of P (  *)  0100011011… L 1 1111001 L 2 0000000 L 3 1000000 L 4 1101000 L 5 1111111 L 6 0010001 L 7 1001111...  strings in {0,1}* 

13. 13 Uncountability of P (  *) Cantor’s Diabolical Diagonal Cantor’s diabolical diagonalization argument takes this supposed table, and creates a language L evil that’s not on the list. I.e., one uses the table to create a row that’s not there proving that the table cannot be comprehensive –contradicting the countability assumption and proving that P (  *) is not countable.

14. 14 Uncountability of P (S*) Cantor’s Diabolical Diagonal L evil is created as follows: The j th column of L evil is the opposite of the j th column of the j th row L i. In other words, L evil is the anti- diagonal of the table. This guarantees that L evil disagrees with every listed language on some string: If the j th language contains the j th string, then L evil doesn’t, and if the j th language doesn’t contain the j th string, then L evil does. Let’s see how this works in practice:

15. 15 Cantor Diagonalization  0100011011… L 1 L 2 L 3 L 4 L 5 L 6 L 7... L evil  strings in {0,1}* 

16. 16 Cantor Diagonalization  0100011011… L 1 1111001 L 2 L 3 L 4 L 5 L 6 L 7... L evil 0  strings in {0,1}* 

17. 17 Cantor Diagonalization  0100011011… L 1 1111001 L 2 0000000 L 3 L 4 L 5 L 6 L 7... L evil 01  strings in {0,1}* 

18. 18 Cantor Diagonalization  0100011011… L 1 1111001 L 2 0000000 L 3 1000000 L 4 L 5 L 6 L 7... L evil 011  strings in {0,1}* 

19. 19 Cantor Diagonalization  0100011011… L 1 1111001 L 2 0000000 L 3 1000000 L 4 1101000 L 5 L 6 L 7... L evil 0110  strings in {0,1}* 

20. 20 Cantor Diagonalization  0100011011… L 1 1111001 L 2 0000000 L 3 1000000 L 4 1101000 L 5 1111111 L 6 L 7... L evil 01100  strings in {0,1}* 

21. 21 Cantor Diagonalization  0100011011… L 1 1111001 L 2 0000000 L 3 1000000 L 4 1101000 L 5 1111111 L 6 0010001 L 7... L evil 011001  strings in {0,1}* 

22. 22 Cantor Diagonalization  0100011011… L 1 1111001 L 2 0000000 L 3 1000000 L 4 1101000 L 5 1111111 L 6 0010001 L 7 1001111... L evil 0110010  strings in {0,1}* 

23. 23 Specifically Undecidable Okay, so now we know that there exist unrecognizable language. In fact, most languages are unrecognizable as the cardinality of P (  *) is a bigger infinity than that of  *! However, the existential proof is not constructive. We would like an explicit example of such a language. THM: A TM is recognizable but undecidable.

24. 24 A TM : Recognizable but Undecidable A TM is recognizable as follows: “On input : 1. Simulate the TM M running on input x 2. If M accepts, then ACCEPT” This algorithm tells us when strings are accepted, but for rejected strings, there is no output, and even worse, for infinite loops, the algorithm gets caught in its own infinite loop as it’s simulating M’s infinite loop.

25. 25 A TM : Recognizable but Undecidable A TM is undecidable: Suppose for contradiction, that A TM were decidable. Then some TM exists, call it D which decides A TM. Consider the implicit encoding scheme M . Use D to construct an evil TM E (in the spirit of Cantor) which we’ll show cannot exist thus proving that A TM must not be decidable.

26. 26 The Evil TM E = “On input w 1. If w doesn’t encode a TM, REJECT. 2. Else, let M be the TM encoded by w. Simulate the machine D running on input. 3. If D accepts, REJECT 4. If D rejects, ACCEPT” Since D is a decider, so is E.

27. 27 The Evil TM The following nagging problem remains: Does E accept or reject the self-encoding ? CASE 1) Accepts. Then D rejected the input >. Therefore, by step 4 of E’s pseudocode, E must not accept. This contradicts the assumption of acceptance! CASE 2) Rejects. Then D accepted the input >. Therefore, by step 3 of E’s pseudocode, E must accept. This contradicts the assumption of rejection! Thus neither case happens so D doesn’t exist!

28. 28 Java Consequences The undecidability of A TM has far reaching consequences. For example: THM (assumes Church-Turing thesis): There is no algorithm for telling if an arbitrary Java program halts. I.e. the language Halt Java = { compiled Java programs that eventually exit on their own } is undecidable.

29. 29 Undecidability of Halt Java Why? Suppose that Halt Java were decidable. Note that there are TM-simulators written in Java. Such a simulator TMSimulate.java would run by, taking two arguments arg1, arg2 as input with arg1 a TM encoding and arg2 a string which the TM will be run on. More specifically, TMSimulate.java contains the method: public static boolean run(String arg1, String arg2) which returns true if and only if M accepts arg2.

30. 30 Undecidability of Halt Java The upshot: The following program would then solve A TM which contradicts the fact that the problem is undecidable (assuming that TM’s can in turn, simulate Java –which follows from the Church-Turing thesis but can also be demonstrated directly).

31. 31 Undecidability of Halt Java public class D{ public static void main(String[] args){ if(TMSimulate.run(args[0],args[1])) System.exit(0); else while(true); } So if an algorithm for telling when Java program halted existed, would be able to tell when TM’s accept by seeing when the program above halts!

32. 32 Undecidability of E TM and EQ TM Recall that E TM is the emptiness problem for Turing machines: “Given a TM M does L(M) =  ?” And that EQ TM is the equality problem: “Given TM’s M and M’ does L(M) = L(M’ )?” Notice that EQ TM at least as hard as E TM since E TM is a special case of EQ TM. Q: Why?

33. 33 Undecidability of E TM and EQ TM A: Just set the second TM M’ to be a TM which rejects all inputs! So to prove that EQ TM is undecidable it suffices to show that E TM is undecidable and we can kill two birds with one stone: THM: E TM is undecidable. Proof. Suppose E TM were decidable. Let F be a decider for E TM. We’ll use F to solve A TM by applying F to a class of TM’s whose purpose is to encode the acceptance problem as instances of the emptiness problem.

34. 34 Undecidability of E TM and EQ TM Given any TM M and string w construct a TM K M,w whose language is nonempty iff w is accepted by M. The description of K M,w is as follows: “On input x 1. Erase x and replace by w. 2. Simulate M running on input w.” So if M accepts w, K M,w accepts every possible x ! On the other hand, if M doesn’t accept w, then K M,w either rejects every x or goes into an infinite loop on every x. Summarizing:

35. 35 Undecidability of E TM and EQ TM If E TM were solvable, we could decide when L(K M,w ) is empty, and therefore decide when M accepts w because of the fact: This completes the proof that E TM and consequently EQ TM are undecidable. The above is a specific case of a general technique for showing that new problems are undecidable by using reductions from previously known undecidable problems.

36. 36 Undecidability of ALL TM The same fact shows that the the problem of telling if a TM accepts all inputs is also undecidable. Formally: ALL TM = “Given a TM M. Does L(M) =  * ?” Q: Is E TM recognizable? Corecognizable? How about ALL TM ?

37. 37 Further Look at E TM A: E TM is corecognizable but not recognizable: We can tell when a TM isn’t empty by simulating the TM on all inputs –by dovetailng– and then outputting NON-EMPTY when the TM accepts some string. E TM is not recognizable because knowing when to answer “YES” would mean that we would know when to answer “NO” for A TM –by the construction above. But this is precisely what we cannot do, since we do know how to answer “YES” for A TM, and being able to also answer “NO” would mean that A TM is decidable.

38. 38 Further Look at ALL TM On the other hand, ALL TM is an example of a language that is neither recognizable nor corecognizable, so is the explicit example of the language proved to exist earlier by the cardinality argument. We cannot answer “NO” to ALL TM as the reverse of the above argument for EQ TM would imply that would be able to answer “NO” to A TM. Thus ALL TM is not corecognizable. Furthermore, ALL TM is also not recognizable as follows:

39. 39 ALL TM : Neither Recognizable nor Corecognizable Suppose we could recognize ALL TM. As TM enumerators convert into TM acceptors, would be able to recognize when a TM enumerates all possible strings. For each TM-string pair (M,w) consider the enumerator E M,w : 1. Simulate M running on w in the background. 2. At the same time, print out  *. 3. If the background simulation accepts, STOP! 4. If it rejects, print out  * forever. By construction, L(E M,w ) =  * iff M doesn’t accept w. Answering “YES” to ALL TM means we can answer “NO” to A TM which is impossible!