1. CS4018 Formal Models of Computation weeks 20-23 Computability and Complexity Kees van Deemter (partly based on lecture notes by Dirk Nikodem)

2. First set of slides: Introduction to Computability Introduction: computability Countably and uncountably infinite sets Enumeration and diagonalisation Some functions must be uncomputable

3. The un-motto of these lectures: Put the right kind of software into a computer and it will do whatever you want it to do. There may be limits on what you can do with the machines themselves, but there are no limits on what you can do with software. (TIME Magazine, 1984. Quoted with abhorrence by Harel, The Science of Computing)

4. Goal of these lectures: sharpen your feeling for the limits on (1) what computers can achieve at all, and (2) what they can achieve given certain constraints on time and space. Side effect: introduce you to some interesting mathematics. (Includes two famous open questions.)

5. Lectures week 20,21 1.Computability: Some problems can be solved by computer, others cannot. Why not? 2.Complexity: –Some programs are faster than others. How can one make this notion precise? –Some problems require programs that take a long time running. How can we tell?

6. Four weeks is a short time: Computability alone could take up an entire course. This necessitates an informal approach, using as little formalism as possible. (Proofs not based on Turing machines etc.) Key insights can still be gained in this way

7. 1.Book Recommendation Algorithmics: the Spirit of Computing, by David Harel The Science of Computing, by David Harel Discrete Mathematics and its Applications, by Kenneth Rosen

8. 2,3,4 What is computability? `Computation is a pre-theoretical notion, The Collins Dictionary and Thesaurus (1987) reflects an outdated view: –computation: a calculation involving numbers or quantities. –calculate: to solve by a mathematical procedure computing scientists have turned it into a technical term

9. Informal explication of `computation Suppose you want to achieve some goal Suppose you have a procedure for doing this (stated in some objective way) Procedure relies on primitives that are unproblematic What makes … a good cooking recipe? good instructions for using data projector?

10. Desirable: (e.g., think of long division) Statement of procedure is explicit, clear and unambiguous; no inventiveness required Statement is finite, and requires finite resources (e.g., money, memory) The outcome of the procedure can be trusted This is called an effective procedure, a computation, or an algorithm.

11. History Gottlob Leibnitz (1700) Calculemus: lets decide all disputes by computation David Hilbert (1900): Can all questions of mathematics be answered algorithmically? Kurt Goedel (1931): This is not possible, even for number theory. (Result was independent of the notion of computation.) Alan Turing (1936): The first formal model of computation, the Turing Machine

12. Other formalisations: –Lambda calculus, –Recursive functions

13. Churchs thesis Proven equivalent: –Turing machines –Lambda calculus –Recursive functions –Any general-purpose programming language Thesis: this is all there is to computation

14. Some variations are still possible How about another requirement: Goal is always achieved in finite time (?)

15. One answer: termination is always necessary This position has the advantage of clarity. Let us look at an example. Here is a simple program for determining whether a number n is prime: (in quasi-PASCAL, as in lecture notes)

16. Problem: is n a prime number? program prime (input, output); var n, i : integer; begin read (n); i := 2; while (i 0) {n is not divisible by i} do i := i+1; if i=n then writeln(n is prime) else writeln(n is not prime); end

17. This procedure is slow, but we will not worry about speed for now (until week 22: complexity)

18. Program terminates. If it did not, it would not be useful. Note: it addresses a (decision) problem: a decision YES/NO What if the goal of the program is to find something? What if that something may not exist? Example: find the smallest prime number that contains a string of n sevens.

19. Find prime containing string of n sevens One approach: Set n, then run previous procedure with increasingly large numbers. For each prime, test whether it contains a string of n sevens, until you have found one. For some n, you may never find one. Does this make the procedure non- effective? Is it not a computation?

20. Some people have answered NO: (second response) Decision problems require termination. Other procedures do not. Both responses are legitimate: The meaning of computation (effective procedure, algorithm) is not cast in stone

21. Another example (Harel) Algorithm A: (1) while x<>1 do x:=x-2; (2) stop Clearly, A terminates iff x is odd

22. Another example (Harel) Algorithm B: (1) while x<>1 do if x is even then do x:=x/2; else do x:=3x+1 (2) stop Try it out with inputs x=3, x=7

23. (1) while x<>1 do if x is even then do x:=x/2; else do x:=3x+1 (2) stop x=3 trace: x:=10,5,16,8,4,2,1 x=7 trace: x:=22,11,34,17,52,26,13,40,20,10,5,16,8,4,2,1

24. B terminates for any known input, but no proof was available as to whether B must always terminate. The strict view implies that we dont know whether B is an algorithm. (We wont take a stand here)

25. Questions about algorithms Does the algorithm terminate? [More about this below] How long does it take? [Weeks 22,23] How much memory does it use? Does the algorithm always achieve the intended goal? [Correctness: other courses]

26. Proof methods Enumeration Diagonalisation Reduction

27. Enumeration Structure of proof: there exist n algorithms there exist m problems m>n --------------------------------- therefore some problems dont have an algorithm

28. Complication: the relevant numbers are large Is it meaningful to ask how many programs there are? - YES Georg Cantor (1845-1918) invented the calculus of infinite numbers. Initially controversial, now very influential

29. Cantors starting point: N = the set of natural numbers = {0,1,2,…}. How many elements? Not 0, not 1, not 2, … Cantor called this number 0 (the set is countably infinite). The next number is called 1 the next one 2 and so on. (Together: uncountably infinite) Question: what is 1 ?

30. Cantors key principle: Sets A and B have the same cardinality iff there exists a 1-1 correspondence between them. (Also called bijection) Let A=N. Then how about 1.B={1,2,3,4,…}? 2.B={1,3,5,7,…}?

31. 1.B={1,2,3,4,…}? Yes, use f:A B such that f(x)=x+1 A: 0 1 2 3 4 5 B: 1 2 3 4 5 6 f is a function, since x A y B(f(x,y)) f(x,y) & f(x,z) implies y=z f is an injection, since f(x,y) & f(z,y) implies x=z f is a surjection, since y B x A(f(x,y))

32. 2.B={1,3,5,7,…}? Yes, B has the same cardinality as N. To see this, use f:A B such that f(x)=2x+1 A: 0 1 2 3 4 5 B: 1 3 5 7 9 11 Note that a 1-1 correspondence with N is an enumeration.

33. So, many sets are equally large as N. They all have cardinality 0 One could easily think that all infinite sets have this cardinality: there would not exist a cardinality 1 This turns out to be false. In fact, the real numbers are a larger set than N. –(In fact, for all sets A, the power set of A is larger than A)

34. Key observation: || || > ||N|| Important for two reasons: 1.We need this fact for our proof by enumeration 2.Proof of || || > ||N|| is seminal Proof proceeds by Reductio Ad Absurdum: By deducing a contradiction from p, we prove that ¬p

35. Here goes: Suppose was countably infinite (i.e., its cardinality is 0 ) Then we could completely enumerate its elements: a1. (first real number) a2. (second real number) a3. (etc.) In decimal notation, real numbers have 0 digits:

36. a1 = a2 = a3 = Now construct a number n(ew)= def n differs from the i-th number on the i-th digit Which of the enumerated numbers is n? n is not a1 because of a 11 n is not a2 because of a 22 Etc. So, n is not in the list! Consequently, the enumeration was not complete after all

37. We have reached a contradiction: the enumeration is complete and incomplete Adding the number n does not help … (Homework: does the argument apply to the set Q of all fractions?) Diagonalisation is a very productive proof method in computability

38. Now back to the first proof method: Enumeration Enumeration is much simpler than diagonalisation. Two questions to ask: (1) How many computations are there? (2) How many things are there to compute? Regarding (1): Lets focus on programs Regarding (2): Lets focus on numerical functions: from integers to integers (called integer functions)

39. Enumeration (1) Regarding simplification (1):computations ~ programs This is a nontrivial step. In computability, people are usually more careful: they work with a formalism that is as simple as possible, while still allowing every reasonable `computation: e.g., Turing machines.

40. Churchs thesis is not an exact result, yet it is practically universally believed Relevance for us: if we prove something about what can be computed using PASCAL or JAVA, then we assume it to hold for any programming language that anyone could think of Hence the simplificationcomputations ~ programs

41. Enumeration (2) Regarding simplification (2): things to calculate ~integer functions This is unproblematic. (If some integer functions are uncomputable then some problems are uncomputable)

42. Noncomputability proof by enumeration: (1)The number of programs is 0. (2)The number of integer functions is 1 (3)Therefore some integer functions cannot be programmed (4)Therefore some integer functions are uncomputable [by Churchs thesis]

43. Noncomputability proof by enumeration: (1)The number of programs is 0. Proof: -The number of programs whose length is 1 character can be enumerated -The number of programs whose length is 2 characters can be enumerated. Etc. -Arrange these enumerations head to tail, and the result is another enumeration

44. Noncomputability proof by enumeration: (2) The number of integer functions is 1 Proof: by diagonalisation: Suppose there existed an enumeration of integer functions: F1, F2, F3, …. Then construct a function N(ew) using the following specification:

45. N differs from F1 in its value for arg. 1 N differs from F2 in its value for arg. 2 Etc. More concretely, we might define N(1)=F1(1)+1 N(2)=F2(2)+1 N(3)=F3(3)+1 Etc.

46. It follows that not all integer functions are computable (Note that the argument can be applied to non-integer functions if we number their arguments and values.)

47. Back to Cantor Cantor proved that || || > ||N|| The proof does not show that nothing may lie in between. The Continuum Problem: –Is any set larger than N but smaller than ? –Equivalent: is || || = 1 ? One of the great open questions of pure mathematics. Nothing computational appears to hinge on it This is not true for the other open problem that we shall meet (week 23)