Lecture 18. Unsolvability Before the 1930’s, mathematics was not like today. Then people believed that “everything true must be provable”. (More formally, in a powerful enough mathematical system, a true statement should be provable as a theorem.) Or put in today’s and computational language: given a computational problem, we should always be able to find a solution to it, in finite amount time, using an expressive programming language say Java. All these had changed because of one man: Kurt Gödel, who has made immense impact upon scientific and philosophical thinking in the 20 th century, and today. Kurt Gödel

You are not that far away from the great man: Me You Anil Nerode Kurt Gödel and Albert Einstein postdoc

Solvability and Decidability Up to now we have been discussing problems that are solvable on a computer, and for each problem we have been trying to find efficient algorithms. Here "solvable" means that there exists a finite, deterministic program given access to unlimited resources in terms of time and space, that will, after a finite time, halt and output the correct solution for the problem. If, further, the computational problem has a "yes/no" answer, we call it a decision problem (If such a problem is solvable, we call them “decidable”. Decision problems are particularly easy to study and analyze. Usually it is possible to take a computational problem and turn it into a related decision problem, and vice versa. One way of doing this is, for example, to add a new parameter i and say "Is the i'th bit of the answer 1 or 0?" By repeatedly asking decision problems of this sort, we could solve a computational problem that has a string or number solution, instead of just true/false.

Is every computational problem solvable? In 1900, David Hilbert gave a famous address at the International Congress of Mathematicians in which he listed 25 important problems for the next century. His 10th problem concerned the solution of Diophantine equations. He asked: given a multivariate polynomial equation with integer coefficients, such as: 3x 2 y + 17 xz - 5 xy = 0. Can you write a computer program (“process”, in Hilbert’s original words) to find solution? You might think you can just do exhaustive search since we do not care about “time complexity”. However, then if no solution exists, your program will run forever.

Russell’s paradox Frege and Russell: Gottlob Frege was a philosopher of mathematics who wanted to axiomatize mathematics ( ); he wanted to write down a system of axioms from which all mathematical truths would follow. Among other things, he was responsible for formalizing the ideas of the existential and universal quantifiers that we still consider essential today. He formulated his ideas in a book called Grundgesetze der Arithmetik (Basic Laws of Arithmetic), but just before the 1903 edition was to go to press, he got a very surprising and disturbing letter from Bertrand Russell. Russell wrote to Frege to show that some sentences which could be constructed according to Frege's rules did not result in a consistent mathematical object. Today this is known as Russell's paradox. To illustrate the idea of Russell's paradox, consider another similar paradox mentioned by Russell (but invented by an unknown acquaintance of him). There is a certain town with a barber. This barber cuts the hair of every person in the town who does not cut their own hair. Who cuts the barber's hair? If the barber cuts his own hair, then according to the rule, he doesn't cut his own hair. And if he doesn't cut his own hair, then by the rule, he does. So we get a contradiction. The only reasonable conclusion is that there cannot be such a barber.

Another paradox: Shortest string that cannot be described in less than thirteen English words. Does this string exist? If it does, then we have just described it above in 12 English words, contradiction. We will use this to prove that we cannot find such a string.

Let’s explain Let's think about sets. Sets contain members; but could we have a set that contained itself as a member? A priori, nothing rules this out; we could, for example let S = the set of all mathematical concepts Since S is itself a mathematical concept, S is a member of S. Another example is S = the set of all ideas expressible in less than 12 words Since S is expressible in less than 12 words, S is a member of S. Frege's axiomatization of mathematics did not rule out expressions such as {S : S ∈ S }. But even worse, it did not rule out expressions such as T = {S : S ∉ S }. Such an expression would denote a valid mathematical object, according to Frege. But as Russell wrote him, this introduces a genuine paradox that is impossible to resolve within Frege's system. The problem comes when we try to decide if T ∈ T. If T ∈ T, then T ∉ T. And if T ∉ T, then T ∈ T. This contradiction so disturbed Frege that he proposed a change to his system that would have created a mathematical universe with only one object.

Randomness (Solomonoff, Kolmogorov, Chatin), 30 years later … in the 1960’s Remember we used “incompressibility” to analyze average case complexity of algorithms. We defined: C(x) = length of the shortest description (program) of x, and this is invariant w.r.t. description language / Turing machine. If C(x) ≥ |x|, i.e. x is not compressible, then we say x is “random”. It turns out that this “randomness” Provides another foundation of mathematics (for probability theory) provides methods for us to analyze algorithms Provides a concrete statement for Godel’s theorem (Godel’s construction was “Theorem: This theorem is not provable”).

Godel’s Theorem in a simple form Theorem. The statement “x is random” is not provable. C(x) > C. Proof (G. Chatin). Let F be an axiomatic theory. C(F)= C. If the theorem is false and statement “x is random” is provable in F, then we can enumerate all proofs in F to find a proof of “x is random” and |x| >> C, output (first) such x. Then C(x) > C. Contradiction. QED

Undecidability Corollary. L = { x | x is random} is not decidable. Proof. If one has a program that, with input x, outputs “yes” iff “x is random”, then this provides a “mathematical proof” (treat the programming language as a mathematical system, inference rules) for “x being random”, contradiction. QED Oh, by the way, Hilbert’s 10 th problem was also proved to be undecidable.

Turing, in 1936 Everyone who taps a keyboard … is working on an incarnation of a Turing machine – Time Magazine In 1928, David Hilbert posed three questions: Is mathematics complete Is mathematics consistent Is mathematics decidable Kurt Gödel answered no to the first two questions in his famous 1931 paper. But for the third, it was actually unclear what do we mean by “decidable” – by what? To make the idea of a program more rigorous, Turing developed the notion of "Turing machine", an abstract computational model that, he argued, could do anything that a human computer could do. He also invented “universal” Turing machine.

Turing’s uncomputability proof Theorem. Halting problem is not decidable. Proof. We prove by contradiction: assume such a program H exists: H(P,x) = 1 if P(x) halts, o.w. H(P,x) = 0. Construct H’ s.t. H’(P) halts iff H(P,P) = 0. Thus, will H’(H’) halt? If it does, then H(H’,H’) =1, hence H’(H’) does not halt. Contradiction. If it does not, then H(H’,H’) = 0, hence H’(H’) halts. Contradiction.