1. Decidable and undecidable problems deciding regular languages and CFL’s Undecidable problems

6. Deciding CFLs Useful to have an efficient algorithm to decide whether string x is in given CFL –e.g. programming language often described by CFG. Determine if string is valid program. Can simulate NPDA, but this takes exponential time in the worst case.

7. Deciding CFLs Convert CFG into Chomsky Normal form. parse tree for string x generated by nonterminal A: A BC x If A  k x (k > 1) then there must be a way to split x: x = yz A → BC is a production and B  i y and C  j z for i, j < k yz

8. Deciding CFLs An algorithm: IsGenerated(x, A) if |x| = 1, then return YES if A → x is a production, else return NO for all n-1 ways of splitting x = yz for all ≤ m productions of form A → BC if IsGenerated(y, B) and IsGenerated(z, C) return YES return NO worst case running time?

9. Deciding CFLs worst case running time exp(n) Idea: avoid recursive calls –build table of YES/NO answers to calls to IsGenerated, in order of length of substring –example of general algorithmic strategy called dynamic programming –notation: x[i,j] = substring of x from i to j –table: T(i, j) contains {A: A nonterminal such that A * x[i,j]}

10. Deciding CFLs IsGenerated(x = x 1 x 2 x 3 …x n, G) for i = 1 to n T[i, i] = {A: “A → x i ” is a production in G} for k = 1 to n - 1 for i = 1 to n - k for k splittings x[i, i+k] = x[i,i+j]x[i+j+1, i+k] T[i, i+k] = {A: “A → BC” is a production in G and B  T[i,i+j] and C  T[i+j+1,i+k] } output “YES” if S  T[1, n] else output “NO”

11. Emptiness problem for CFL’s L = { | L(G) is empty } We can show that L is decidable using the following algorithm: Input: A CFG G. Output: ‘yes’ if L(G) is empty, ‘no’ else. N1 = { }; N2 = { A | A -> w is a rule in G for some w in * } While (N1 != N2) { N1 = N2; N2 = { A | A -> for some in (  U N1)* } } if (S is included in N1) output yes else output no;

12. Problems related to CFG Equivalence problem for CFG Given two CFG’s G1 and G2, determine if L(G1) = L(G2). It turns out that this problem is undecidable. This means, it is impossible to design an algorithm to solve this problem.

13. 13 Countable and Uncountable Sets Nthe natural numbers N = {1,2,3,…} are countable Definition: a set S is countable if it is finite, or it is infinite and there is a bijection N f: N → S

14. 14 Countable and Uncountable Sets Theorem: the positive rational numbers N Q = {m/n : m, n  N } are countable. Proof: 1/1 1/2 1/3 1/4 1/5 1/6 … 2/1 2/2 2/3 2/4 2/5 2/6 … 3/1 3/2 3/3 3/4 3/5 3/6 … 4/1 4/2 4/3 4/4 4/5 4/6 … 5/1 … …

15. Countable and Uncountable Sets R Theorem: the real numbers R are NOT countable (they are “uncountable”). How do you prove such a statement? –assume countable (so there exists bijection f) –derive contradiction (some element not mapped to by f) –technique is called diagonalization (Cantor)

16. 16 Countable and Uncountable Sets Proof: R –suppose R is countable R –list R according to the bijection f: nf(n) _ 13.14159… 25.55555… 30.12345… 40.50000… …

17. 17 Countable and Uncountable Sets Proof: R –suppose R is countable R –list R according to the bijection f: nf(n) _ 13.14159… 25.55555… 30.12345… 40.50000… … set x = 0.a 1 a 2 a 3 a 4 … where digit a i ≠ i th digit after decimal point of f(i) (not 0, 9) e.g. x = 0.2312… x cannot be in the list!

18. 18 non-RE languages Theorem: there exist languages that are not Recursively Enumerable. Proof outline: –the set of all TMs is countable –the set of all languages is uncountable –the function L:{TMs} →{languages} cannot be onto

19. 19 non-RE languages Lemma: the set of all TMs is countable. Proof: –each TM M can be described by a finite-length string N –can enumerate these strings, and give the natural bijection with N

20. 20 non-RE languages Lemma: the set of all languages is uncountable Proof: –fix an enumeration of all strings s 1, s 2, s 3, … (for example, lexicographic order) –a language L is described by its characteristic vector  L whose i th element is 0 if s i is not in L and 1 if s i is in L

21. 21 non-RE languages –suppose the set of all languages is countable –list characteristic vectors of all languages according to the bijection f: nf(n) _ 10101010… 21010011… 31110001… 40100011… …

22. 22 non-RE languages –suppose the set of all languages is countable –list characteristic vectors of all languages according to the bijection f: nf(n) _ 10101010… 21010011… 31110001… 40100011… … set x = 1101… where i th digit ≠ i th digit of f(i) x cannot be in the list! therefore, the language with characteristic vector x is not in the list

23. 23 The Halting Problem Definition of the “Halting Problem”: HALT = { : TM M halts on input x } HALT is recursively enumerable. – proof? Is HALT decidable?

24. 24 The Halting Problem Theorem: HALT is not decidable (undecidable). Proof: –Suppose TM H decides HALT –Define new TM H’: on input if H accepts > then loop if H rejects > then halt

25. 25 The Halting Problem Proof: –define new TM H’: on input if H accepts > then loop if H rejects > then halt –consider H’ on input : if it halts, then H rejects >, which implies it cannot halt if it loops, then H accepts > which implies it must halt –contradiction.

26. 26 The Halting Problem Turing Machines inputs Y n Y n n Y n YnYYnnY H’ : box (M, x): does M halt on x? The existence of H which tells us yes/no for each box allows us to construct a TM H’ that cannot be in the table.

27. 27 So far… Can we exhibit a natural language that is non-RE? regular languages context free languages all languages decidable RE {a n b n : n ≥ 0 } {a n b n c n : n ≥ 0 } some language HALT

28. 28 RE and co-RE The complement of a RE language is called a co-RE language regular languages context free languages all languages decidable RE {a n b n : n ≥ 0 } {a n b n c n : n ≥ 0 } some language HALT co-RE

29. 29 RE and co-RE Theorem: a language L is decidable if and only if L is RE and L is co-RE. Proof: () we already know decidable implies RE –if L is decidable, then complement of L is decidable by flipping accept/reject. –so L is in co-RE.

30. 30 RE and co-RE Theorem: a language L is decidable if and only if L is RE and L is co-RE. Proof: () we have TM M that recognizes L, and TM M’ recognizes complement of L. –on input x, simulate M, M’ in parallel –if M accepts, accept; if M’ accepts, reject.

31. 31 A natural non-RE language Theorem: The complement of HALT is not recursively enumerable. Proof: –we know that HALT is RE –suppose complement of HALT is RE –then HALT is co-RE –implies HALT is decidable. Contradiction.

32. Summary Main point: some problems have no algorithms, HALT in particular. regular languages context free languages all languages decidable RE {a n b n : n ≥ 0 } {a n b n c n : n ≥ 0 } some language HALT co-RE co-HALT