1. Turing’s Thesis: The Universal Machine
4/30/2019
Turing’s Thesis: The Universal Machine
CSN9902Turing

2. Turing and the Fundamental Limits of Computing
Aims
To explore the roots of the field of computing
Show the basic concept of a computer
Discuss the fundamental limitations of computing
Why
Show you the intellectual roots of your discipline
Pure fun and stimulation
Knowledge of knowledge
Insight into yourself and your world
Iam nova progenies caelo demittitur alto†
Virgil Ecolugues IV
†A new generation now descends from above
30/04/2019

3. Why? - Ignorance
“There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know”.
Donald Rumsfeld, DoD news briefing Feb. 12, 2002,
0th Order Ignorance (0OI) — Lack of Ignorance. I can demonstrate my lack of ignorance in some tangible form, 0OI is knowledge, When I have 0OI, I have the answer to the problem.
1st Order Ignorance (1OI) — Lack of Knowledge. I have 1OI when I don’t know something and can readily identify that fact. 1OI is basic ignorance. I know what I don’t know. When I have 1OI, I have the question. Usually, having a good question makes it fairly easy to find the answer.
2nd Order Ignorance (2OI) — Lack of Awareness. I have 2OI when I don’t know that I don’t know something. That is to say, not only am I ignorant of something (for instance I have 1OI), I am unaware of this fact. I don’t know enough to know that I don’t know enough. This is the real problem. Not only do I not have the answer I need, I don’t even have the question.
3rd Order Ignorance (3OI) — Lack of Process. I have 3OI when I don’t know a suitably efficient way to find out I don’t know that I don’t know something. This is lack of process, and it presents a major problem: If I have 3OI, I don’t know of a way to find out there are things I don’t know that I don’t know. Therefore, I can’t change those things I don’t know that I don’t know into either things that I know, or at least things I know that I don’t know, as a step toward converting the things I know that I don’t know into things I know.
4th Order Ignorance (4OI) — Meta Ignorance. I have 4OI when I don’t know about the Five Orders of Ignorance.
30/04/2019
Phillip G. Armour “The Five Orders of Ignorance. Viewing software development as knowledge acquisition and ignorance reduction”. CACM V43(10):17..20

4. Why? - Reason
A Vulgar Mechanick can practise what he has been taught or seen done, but if he is in error he knows not how to find it out and correct it, ...; Whereas he that is able to reason nimbly and judiciously ... is never at rest till he gets over the rub”
Isaac Newton
30/04/2019

5. 4/30/2019
Summary
Philosophical and Historical Background - Search for truth – the concept of proof - Cantor, Hilbert and Gödel
The Turing Machine
The Church-Turing Thesis
The Halting Problem
Fundamental limits of computation
Incomputable/Insolvable
Intractability
Physical limits
30/04/2019
CSN9902Turing

6. Background Concept of Proof Use of Aristotelian logic
4/30/2019
Background
Concept of Proof
Use of Aristotelian logic
Transfinite Cardinality and Countability, Cantor 1877
Hilbert’s Unsolved Problems – Hilbert 1900
The Principia Mathematica – Whitehead & Russell 1910
On Formally Undecidable Propositions of the Principia Mathematica – Gödel 1931
On Computable Numbers with an Application to the Entscheidungsproblem – Turing 1936
30/04/2019
CSN9902Turing

7. How do we know something is true?
Is truth whatever is the consensus that we all agree on. Is there an objective truth or is truth a social/cultural construct
Is truth something that corresponds to reality?
Empirically testable - “All swans are white”
Is truth dependant on semantics and interpretation?
“This sentence is not true”.
Are there statements that are true irrespective of reality?
“(P   P)”
30/04/2019

8. Key Elements of Scientific Proof
Objective
So scientist does not bias interpretation of results or change results
Control experiments - to show what would have happened if nothing had been done
Replicable/repeatable
Verifiable measurement and comparison
The reproducibility of experimental results by other researchers
Predictive
Of currently unknown outcomes – may be statistical and only talk about probabilities.
Known outcomes are called consequences.
Falsifiable
Elimination of plausible alternatives
Whatever the consensus it must in principle remain tentative
Hard to vary explanations
Causal
Identification of causes, correlation of events, time-order relationships
30/04/2019 8

9. OUTPUT (MEASUREMENTS)
The Wheeler Cycle
Theoretical Model
PREDICTIVE OUTPUT
SET-UP PARAMETERS
EXPERIMENT
INPUT DATA
OUTPUT (MEASUREMENTS)
Physical Process
30/04/2019 9

10. Logical Argument P  ¬P eg. (I am wealthy) OR (I am NOT wealthy)
If I owned all the money in Lloyds Bank I would be wealthy I do not own all the money in Lloyds Bank
Therefore I am not wealthy
Valid facts, invalid argument, true conclusion
All managers are female All females are patient
Therefore all managers are patient
Invalid facts, valid argument, false conclusion
P  ¬P eg. (I am wealthy) OR (I am NOT wealthy)
tertium non datur, True irrespective of facts
If the Queen owned all the money in Lloyds Bank she would be wealthy
The Queen does not own all the money in Lloyds Bank
Therefore she is not wealthy.
Valid facts, invalid argument, false conclusion
All whales are mammals
All mammals have lungs
Therefore all whales have lungs
Valid facts, valid argument, true conclusion
¬(P Ù ¬P)
eg. It is NOT true that ((I am wealthy) AND (I am NOT wealthy))
30/04/2019

11. Example of Formal Proof Pythagoras's Proof of the Irrationality of √ 2
Suppose √2 = p/q
where
p and q  N
and p and q are in lowest terms (i.e. no common factor)
i.e. 2 = p2/q2
p2 = 2q2
p must be even (as odd  odd = odd)
If p is even then it is of the form p = 2r
so (2r)2 = 2q2
4r2 = 2q2
q2 = 2r2
q2 is even and therefore q must be even
So p and q are both even
but we have already asserted that p and q have no common factor
we are thus in a logical bind.
30/04/2019

12. Greg Cantor and Numbers
4/30/2019
Greg Cantor and Numbers
Cantor defined counting by pairing.
Take two sets.
Take one element out of one and one out of the other
Repeat until either:
Neither set has anything left in it (in which case they have the same cardinality) or
One set is empty and the other has something left in it (in which case the one with something left is larger than the one without anything)
If you construct a set with a known cardinality (e.g. ‘3’) then you can count if another set has three element by pairing against that first set
30/04/2019
CSN9902Turing

13. Cantor’s Transfinite Numbers
4/30/2019
Cantor’s Transfinite Numbers
But let us remember that we are dealing with infinities and indivisibles, both of which transcend our finite understanding...
Galileo Galilei
A set S is called countable if there exists a surjective function
f: º £ªS
If f is even bijective then S is called countable infinite or denumerable.
Note a set is finite if its cardinality is the same as its ordinality.
Aleph-null
Aleph-null (א0) is by definition the cardinality of the set of all natural numbers, and is the smallest of all infinite cardinalities. A set has cardinality א0 if and only if it is countably infinite, which is the case if and only if it can be put into a direct one-to-one correspondence with the integers. Such sets include the set of all prime numbers and the set of all rational numbers.
30/04/2019
CSN9902Turing

14. Counting the Rationals
4/30/2019
Counting the Rationals
Cantor’s proof that the set of rationals is the same size as the set of natural numbers by pairing
1/1
1/2
1/3
1/4
1/5
1/6
1/7
1/8 …
2/1
2/2
2/3
2/4
2/5
2/6
2/7
2/8
3/1
3/2
3/3
3/4
3/5
3/6
3/7
3/8
4/1
4/2
4/3
4/4
4/5
4/6
4/7
4/8
5/1
5/2
5/3
5/4
5/5
5/6
5/7
5/8
6/1
6/2
6/3
6/4
6/5
6/6
6/7
6/8
7/1
7/2
7/3
7/4
7/5
7/6
7/7
7/8
8/1
8/2
8/3
8/4
8/5
8/6
8/7
8/8 1 2 6 7 15 16 28 29 … 3 5 8 14 17 27 30 4 9 13 18 26 31 10 12 19 25 32 11 20 24 33 21 23 34 22 35 36
30/04/2019
CSN9902Turing

15. Counting the Real Numbers
4/30/2019
Counting the Real Numbers
Cantor's original proof shows that the interval (0,1] is not countably infinite.
The proof by contradiction proceeds as follows:
(1) Assume (for the sake of argument) that the interval (0,1] is countably infinite.
(2) We may then enumerate all numbers in this interval as a sequence, ( r1, r2, r3, ... )
(3) We already know that each of these numbers may be represented as a decimal expansion.
(4) We arrange the numbers in a list (they do not need to be in particular order). Assume, for example, that the decimal expansions of the beginning of the sequence are as follows:
r1 = r2 = r3 = r4 = r5 = r6 = r7 =
30/04/2019
CSN9902Turing

16. (6) From these digits we define the digits of x as follows.
4/30/2019
(5) We shall now construct a real number x in [0,1] by considering the kth digit after the decimal point of the decimal expansion of rk.
r1 = r2 = r3 = r4 = r5 = r6 = r7 =
The digits we will consider are underlined and in bold face, illustrating why this is called the diagonal proof.
(6) From these digits we define the digits of x as follows.
if the kth digit of rk is 9 then the kth digit of x is 0 otherwise the kth digit of x is the kth digit of rk plus 1
For the example above this will result in the following decimal expansion. x =
(7) The number x is a real number (we know that all decimal expansions represent real numbers) in [0,1] (clearly).
(8) Hence we must have rn = x for some n, since we have assumed that (r1, r2, r3, ... ) enumerates all real numbers in [0, 1].
However, because of the way we have chosen digits in step (6), x differs in the nth decimal place from rn, so x is not in the sequence (r1, r2, r3, ... ) .
This sequence is therefore not an enumeration of the set of all reals in the interval [0,1]. This is a contradiction.
(11) Hence the assumption (1) that the interval [0,1] is countably infinite must be false.
30/04/2019
CSN9902Turing

17. When old age shall this generation waste,
4/30/2019
When old age shall this generation waste,
Thou shalt remain, in midst of other woe
Than ours, a friend to man, to whom thou say'st,
Beauty is truth, truth beauty,’ - that is all
Ye know on earth, and all ye need to know.”
  John Keats, Ode on a Grecian Urn
30/04/2019
CSN9902Turing 17

18. Hilbert’s Grand Challenge
4/30/2019
Hilbert’s Grand Challenge
“Mathematical Problems”
Lecture delivered before the International Congress of Mathematicians at Paris in 1900, By Professor David Hilbert
Given their mechanistic concepts of science and engineering maths, the Victorians grew to believe everything was knowable and do-able through the power of mathematics.
To mark the start of the 20th Centaury Hilbert (arguably the greatest mathematician of the last hundred years) was invited to give a talk enumerating the last remaining unsolved problems
30/04/2019
CSN9902Turing

19. Hilbert’s 23 Unsolved Problems
4/30/2019
Hilbert’s 23 Unsolved Problems
Number 2: To prove that they [the axioms of arithmetic] are not contradictory, that is, that a definite number of logical steps based upon them can never lead to contradictory results.
Number 11: To devise a process according to which it can be determined by a finite number of operations whether [a diophantine] equation is solvable in rational integers.
In other words: Is there a method for establishing the truth or falsity of any statement written using mathematics/formal predicate logic (the “entscheidungsproblem” (decision problem))?
1. Is mathematics complete? i.e. can every statement either be proved or disproved?
2. Is mathematics consistent? i.e. a statement and its negation cannot be both be provable?
3. Is mathematics decidable? i.e. does there exist a definite (finite mechanical) method (algorithm) which could, in principle be applied to any assertion, and which was guaranteed to produce a correct decision as to whether the assertion was true?
30/04/2019
CSN9902Turing

20. Peano’s Axioms ╒PEANO [X] ══════════ │ z : X │ f : X ¬ X
4/30/2019
Peano’s Axioms
╒PEANO [X] ══════════
│ z : X
│ f : X ¬ X
├─────────────
│ z  ran f
│ A : ¡ X 
│ z  A  f Ó A Ô  A  A = X
└──────────────────
An interpretation: (with X interpreted as: ‘natural number’, z: ‘zero’ and f: ‘successor’)
Zero is a natural number
• The successor of any natural number is another natural number
• There are no two natural numbers with the same successor
• Zero is not the successor of a natural number
• Every property of zero, which belongs to the successor of every number with this property belongs to all numbers
30/04/2019
CSN9902Turing

21. 4/30/2019
Gödel’s Theorem
With only the assumption that Peano’s axioms are true, Gödel proves that the entscheidungsproblem cannot be solved:
“Let k be any recursive consistent class of formulas; then the sentential formula stating that k is consistent is not k-provable; in particular, the consistency of P is not provable in P, provided P is consistent”.
Gödel, K., (1931) On Formally Undecidable Propositions of the Principia Mathematica,
Reprinted in Frege and Gödel, Two Fundamental Text in Mathematical Logic, J. van Heijenort, Harvard University Press, 1967
30/04/2019
CSN9902Turing

22. 4/30/2019
Turing’s Thesis
Gödel’s Theorem shows the entscheidungsproblem is unsolvable and hence the fundamental limits of what is knowable. But has a flaw: it assumes a mathematician.
Turing solves this problem by providing a mathematical definition of what is a mathematician!
In his paper he provides three proofs in which he constructs a model of a mathematician then applies it to the entscheidungsproblem.
We will just do the ‘intuitive construction’ of the Turing machine.
30/04/2019
CSN9902Turing

23. On Computable Numbers with an Application to the Entscheidungsproblem
4/30/2019
The Turing Machine
On Computable Numbers with an Application to the Entscheidungsproblem
A single machine which can be used to compute any computable sequence.
Consists of:
A tape
An finite alphabet of symbols
A set of states
A read/write head which can move along the tape.
Alan. M.Turing, (1936), “On Computable Numbers, With an Application to the Entscheidungsproblem”, Proc. London Math. Soc., 2(42): ; “A correction” ibid, 2(43):
30/04/2019
CSN9902Turing

24. Intuitive Construction of the Machine:
4/30/2019
Intuitive Construction of the Machine:
“Computing is normally done by writing certain symbols on paper. We may suppose this paper is divided into squares like a child’s arithmetic book”.
Do the squares have to be laid out in two dimensions? No - we can cut the paper into strips, sellotape them together and use a one dimensional tape instead. 3 = 2 +
30/04/2019
CSN9902Turing

25. The Alphabet Should be finite. Why?
4/30/2019
The Alphabet
Should be finite. Why?
If there was an infinity of symbols there would be symbols differing to an arbitrarily small extent.
How large does the alphabet have to be?
Doesn’t matter as it is always possible to use sequences of symbols in place of single symbols. Thus a numeral such as 17 or is normally treated as a single symbol.
If we only had two symbols (e.g. ‘0’ and ‘1’) we could use binary to represent the numeral 5 by the sequence ‘101’ or the character ‘x’ by a sequence such as ‘ ’
Are these the same?
Is this a ‘1’ or a ‘7’?
30/04/2019
CSN9902Turing

26. Reading and Writing Symbols
4/30/2019
Reading and Writing Symbols
How much can you read in one go?
If compound symbols are too lengthy they cannot be observed in one glance. e.g. we cannot tell at a glance whether and are the same.
We must make successive observations and change our ‘state of mind’ as we do so.
Providing we can change state as we read symbols we only need to be able to read or write one symbol at a time.
30/04/2019
CSN9902Turing

27. Scanning through the Tape
4/30/2019
Scanning through the Tape
When we are working on paper we rarely read it sequentially, often we need to jump around, for example if we are reading a reference book we may skip to a particular chapter find a particular paragraph which we read and maybe go to the previous paragraph or index to check something. Similarly if you are calculating something on paper you often skip to a blank page to do some side calculations or go back to get a previous result and then return to where you were.
How far from our current position on the tape do we ever need to be able to jump in one go?
Actually we only ever need to move just one square to the right or one to the left. If we need to go further than we can simulate this by moving the one square several times.
30/04/2019
CSN9902Turing

28. Summary of Turing Machine Operations
4/30/2019
Summary of Turing Machine Operations
The most general single operation is therefore:
1. Read the symbol (1, 0 or blank) on the observed square
Depending on your current state of mind:
2. Write a symbol on the square (if you do not want to change the symbol on the square write back what you have just read).
3. Move left or right one square or if you have finished simply halt.
Finally depending on what you read and what state you were in you may want to change your state of mind to a new one.
So a program can be simply represented as a set of quintuples: symbol read, current state, symbol written, next state, direction to move
30/04/2019
CSN9902Turing

29. Alex’s Model of a Turing Machine
4/30/2019
Alex’s Model of a Turing Machine
tape
Read/Write head 1 1 1
Current State: 0
current symbol symbol next move state read write state L
L R
1 0 1 Halt -
current state
Program quintuples
30/04/2019
CSN9902Turing

30. Example TM Program – A Parity Counter
4/30/2019
Example TM Program – A Parity Counter
Initial tape: string of 1’s delimitated by an A on one side and a B on the other:
A B
current
state:
Initial state is 0 and starting position has read/write head over the ‘A’ on the tape
30/04/2019
CSN9902Turing

31. Parity Counter Quintuples
4/30/2019
Parity Counter Quintuples
String of 1s bounded by an ‘A’ and a ‘B’. Tape head starts on the ‘A’ at the left end of the string
Old Symbol New Symbol Direction
State read state written of motion
0 A 0 A R R R
0 B halt 0 -
1 B halt 1 -
If the original string contained an odd number of 1’s, B will be replaced by 1; and even number B will be replaced by 0.
30/04/2019
CSN9902Turing

32. Relationship between TMs and TGs
4/30/2019
Relationship between TMs and TGs
A set of quintuples for a Turning machine can be represented as a transducer state transition graph with the nodes as states and the arcs labelled with read inputs and write and move outputs. e.g. the parity checker:
B / 0; Halt
state 0
state 1
1 / 1; R
B / 1; Halt
A / A; R
30/04/2019
CSN9902Turing

33. Example 2 – A Parenthesis Checker
4/30/2019
Example 2 – A Parenthesis Checker
Output 1 if string is well formed (eg. “(()(()))”)
otherwise output a ‘0’ (eg. “)()(()”).
Old Symbol New Symbol Direction
State read state written of motion
0 A 0 A R
0 ) 1 X L
0 ( 0 ( R
0 B 2 B L
0 X 0 X R
1 ) 1 ) R
1 ( 0 X L
1 A halt 0 -
1 X 1 X L
2 ( halt 0 -
2 A halt 1 -
2 X 2 X L
30/04/2019
CSN9902Turing

34. The Parenthesis Checker as a FSM
4/30/2019
The Parenthesis Checker as a FSM
B / B; L
state 0
state 1
) / X; L
( / X; L
A / 0; Halt
A / A; R
state 2
( / (; R
) / ); R
A / 1; Halt
( / 0; Halt
X / X; L
X / X; R
30/04/2019
CSN9902Turing

35. The Formal Definition M á á Q, S, T, P, q0, F ñ where:
4/30/2019
The Formal Definition
M á á Q, S, T, P, q0, F ñ
where:
Q is a finite set of states (generally denoted by natural numbers)
S is a finite set of symbols one of which is denoted by Ù and is called the blank symbol
T Í S \ { Ù } is the set of input symbols (normally just 1 & 0)
P is a program which is the partial function
(Q \ F) × S © Q × S × { L, R, 0 } (Where L = move left, R = move right, 0 = halt)
q0 Î Q is the initial state
F Í Q is a set of final states
30/04/2019
CSN9902Turing

36. 4/30/2019
Von Neumann Machines
A von Neumann machine implements the formal definition of a Turing Machine in terms of physical switches, relays etc.
Standard differences:
Instead of reading one bit at a time a 32 bit machine reads 32, a 64 bit machine reads 64 etc.
Instead of having to move through the tape (memory) one square at a time a von Neumann machine has addressable random access memory. (Actually until recently machines relied a lot on sequential access memory).
Some bits of the tape are mapped to input and output devices
30/04/2019
CSN9902Turing

37. Variations on the Theme
4/30/2019
Variations on the Theme
All variations on the basic Turing Machine have been shown to be simulateable on a TM
Unidirectional tapes
Multihead/multitape Turing machines (Post machines)
Nondeterministic machines i.e. where given the same input and state a machine has a number of options. e.g. quintuples for a nondeterministic machine (what will it do if it is in state 1 and reads a B?)
Old State Symbol read New state Symbol written Direction
0 A 1 A R
1 B 1 A R
1 B 0 B R
1 B halt C -
…..
30/04/2019
CSN9902Turing

38. The Church-Turing Thesis
4/30/2019
The Church-Turing Thesis
“Every effectively computable function is Turing computable.”
“There are no functions that can be defined by humans, whose calculation can be described by any well defined algorithm that people can be taught to perform that cannot be computed by Turing machines. The Turing machine is believed to be the ultimate calculating mechanism.”
“Whereof one cannot speak, thereof one must be silent.”
L. Wittgenstein, Tractatus Logico-Philosophicus
30/04/2019
CSN9902Turing

39. 4/30/2019
A Side Note
Simple FSAs can be shown to be equivalent to Chomsky Type 3 (Regular) Grammars, i.e. all productions are of the form: B ® tC or B ® t
where B and C are nonterminal tokens and t is a terminal
This reveals the limitations of FSAs due to the theorem “There is a type 2 (context free) language which has no type 3 grammar” (e.g. {0n, 1n : n Î £ })
TMs can be shown to be equivalent to Type 0 (recursively enumerable) grammars (i.e. no restriction on productions). Therefore anything that can be expressed with a type 0 grammar can be computed on a Turing machine. (Kleene’s theorem)
30/04/2019
CSN9902Turing