CIS 5513 - Automata and Formal Languages – Pei Wang 9. Undecidability CIS 5513 - Automata and Formal Languages – Pei Wang

What can a computer do? What language can be defined by any computational device whatsoever? “… recognizing the strings in a language is a formal way of expressing any problem, and solving a problem is a reasonable surrogate for what it is that computers do.”  textbook page 315

Problems and languages (review) In mathematics and computer science, a “problem” is often defined as a function from input to output In principle, a “function” problem can be converted into a “decision” problem that only has “yes/no” answers So to solve a problem means to decide whether a given string belongs to a language

Decidable problems A problem is decidable if and only if there is an algorithm that always tells correctly whether an instance of the problem has answer “yes” or “no”; If no such algorithm exists, the problem is undecidable Decidability corresponds to a universal proposition on problem instances, while undecidability to an existential proposition

Decidability and computability A function is computable if and only if there is an algorithm (or TM) that computes the output of the function with any valid input; If no such algorithm exists, the function is uncomputable The computability of a function problem can be converted to the decidability of a corresponding decision problem

Recursive enumerability A language is recursively enumerable (RE) if its sentences are accepted by a TM A language L is recursive if a TM not only accepts its sentences, but also rejects its non- sentences in finite steps, that is, both L and its complement are RE, and L is decidable If a language L is not recursive, then it is undecidable

Reduction and decidability A problem P1 is reducible to a problem P2 if an algorithm for solving problem P2 could also be used to solve problem P1 “P1 is reducible to a problem P2” means P1 corresponds to a special type of P2. In this case, if P2 is decidable, so is P1; if P1 is undecidable, so is P2 A problem’s undecidability is usually proved by reducing a known undecidable problem to it

Undecidable problems A problem is undecidable if there cannot be any algorithm for it (with guaranteed halting). It is different from a problem for which no algorithm has been found Since the algorithms (TMs) are countable, while the problems (languages) are uncountable, there must be undecidable problems Countability and Cantor's diagonal argument

A binary word as a TM Every TM can be coded as a finite binary string Every finite binary string can be interpreted as a TM according to the above coding method, or as a default TM that accepts { } All finite binary strings can be enumerated in a table, in which each entry corresponds to (1) a word, (2) a TM, and (3) a RE language All TMs are in this table

A language that is not RE In the (infinitely long) table of {0, 1}*, for every index i, word wi can be interpreted as a TM Mi Define Ld = {wi | wi is not in L(Mi)} If there is a j such that Ld = L(Mj) Then wj is in L(Mj)  wj is in Ld  wj is not in L(Mj) So Ld cannot be in the table and cannot be RE

A RE language that is not recursive The universal language Lu = {(M, w) | w is in L(M)} is RE, as it can be accepted by the universal TM U, which simulates M’s reaction to w Lu is not recursive, otherwise its complement Luc would be recursive, too, but Ld can be reduced to Luc by mapping its element w into (w, w) Membership of a RE language is undecidable

Undecidable problems of RE Every nontrivial (i.e., neither none nor all) property of RE languages is undecidable, e.g. Whether a RE language is empty Whether a RE language is finite Whether a RE language is regular Whether a RE language is context-free Proof: Lu can be reduced to each of them Whether a CFG is ambiguous is undecidable, too

The halting problem If all problems are decidable, then there is a program P(x, y) that can judge whether any program x halts on any input y In that case another program Q(x) can be built to halt if and only if P(x, x) returns No However, Q(Q) halt  P(Q, Q) returns No  Q(Q) does not halt Therefore, no such a P(x, y) can exist

Language hierarchy with examples Regular: {0*1*} Context-free: {0n1n} Recursive (decidable): {0n1n2n} Recursively Enumerable: Lu = {(M, w) | w is in L(M)} Non-RE: Luc = {(M, w) | w is not in L(M)}

Motivation and implication Hilbert’s program: to find axioms for all mathematics, and algorithms to decide the truth-value of any mathematical statement Gödel's incompleteness theorem: no consistent system of axioms can prove all truths about the relations of the natural numbers Turing's undecidability theorem: no general algorithm can solve the halting problem for all possible program-input pairs

Self-reference with negation Liar paradox Cantor's diagonal argument Russell's paradox Gödel's incompleteness theorem Turing’s halting problem For a general discussion, see Gödel, Escher, Bach: An Eternal Golden Braid