1. Cognitive Computing 2012-2013 Consciousness and Computation: computing machinery and intelligence 1. TURING MACHINES Mark Bishop

2. 01/04/2014(c) Bishop: Consciousness and Computation2 Why is a theory of computation important? Because the theory of computation impacts: The fundamentals of computing The formalisation of just what is meant be the notion of computing. The fundamentals of mathematics If provides insight into the failure of Hilberts program: the formalisation of all mathematics in axiomatic form and proof that this form is consistent. The fundamentals of Artificial Intelligence and Cognitive Science The ideology that the mind/psychology is fully explainable in computational terms.

3. 01/04/2014(c) Bishop: Consciousness and Computation3 The algorithm (historical) 9thC. Abū ʿ Abdallāh Mu ammad ibn Mūsā al-Khwārizmī, (a Persian scholar based in the House of Wisdom in Bagdad) Amongst other things Al-Khwarizmi outlined a finite series of steps, (Al-jabr, or transformations; algebra), which when followed produce the (real) roots of quadratic equations. In so doing Al-Khwarizmi, (or Algorismus as he was known in Latin), had not only spoken of, but also given his own name to, the new art of computation; the algorithm.

4. 01/04/2014(c) Bishop: Consciousness and Computation4 The algorithm (modern) A modern definition states that a procedure to achieve a specified result is an algorithm iff: Each step is blind, (each step needs no further insight). Steps follow on blindly, (no insight needed to determine the next step). A result is guaranteed after a finite number of steps, (cf. WINDOWS).

5. 01/04/2014(c) Bishop: Consciousness and Computation5 On computers … Originally the term computer did not refer to a machine, but a human being :- A mathematical assistant who calculated by rote in accordance with someeffective method. A method is effective iff: … it demands no insight or ingenuity from the human computer. … produces a result in a finite number of steps. It is clear that the notion of effective method is closely linked with that of, the algorithm.

6. 01/04/2014(c) Bishop: Consciousness and Computation6 On computing machines … A computing machine was originally simply a term used to describe a machine, used by a human computer, which automated some aspect(s) of the calculations required when following an effective method to solve some task. Hence the use of computing machines is analogous to that of modern [non-programmable] calculators. However, in principle, the operation of a computing machine could always be fully performed by a human computer following an effective method.

7. 01/04/2014(c) Bishop: Consciousness and Computation7 What is an effective method? Because neither insight or ingenuity are defined, the mathematical notion of effective method is not rigorously defined. Turing set out to provide a formal model of what can be achieved by a human computer following an effective method by rote. To the question 'what is a "mechanical" process?' Turing returned the characteristic answer, 'Something that can be done by a machine' and he embarked on the highly congenial task of analysing the general notion of a computing machine. In the specification of the Turing Machine, (TM), Turing outlined his idealisation of the operation of a Human Computer following an effective method.

8. 01/04/2014(c) Bishop: Consciousness and Computation8 Finite State Automata A finite-state automaton (plural: automata) or finite-state machine (FSM), or simply [discrete] state machine, is a simple mathematical abstraction, sometimes used to design digital logic or computer programs. It is a behavior model composed of a finite number of states, transitions between those states, and actions. It has finite internal memory, an input feature that reads symbols in a sequence, one at a time without going backward; and an output feature, (which may be in the form of a user interface, once the model is implemented). The operation of an FSM begins from one of the states (called a start state), goes through transitions depending on input to different states and can end in any of those states available … … Only a certain set of states mark a successful flow of operation (called ACCEPT states).

9. Turings input-less discrete State Machine (1950) At each time step the machine occupies one of a finite number of (3) possible physical positions. In operation it cycles through a finite number of (3) possible computational states, {Q 1, Q 2, Q 3 }. The next computational state is modally determined by the current state. The output {LIGHT-ON} occurs when the machine is in computational state Q 1. 01/04/2014(c) Bishop: Consciousness and Computation9

10. FSA specification Three sets and two functions attached to state machines. FSAs can be described by the five tuple IFinite set of inputs QFinite set of internal states ZFinite set of outputs aInput mapping function:f a (Q, I) Q bMoore output function:f b (Q) Z bMealy output function:f b (Q, I) Z The Moore and Mealy forms of FSA are equivalent. 01/04/2014(c) Bishop: Consciousness and Computation10

11. 01/04/2014(c) Bishop: Consciousness and Computation11 Classification of FSAs There are two groups distinguished: Acceptors (or Recognizers) and Transducers. Acceptors (recognizers) produce a binary output, indicating if the FSA input is accepted by the machine or not. All states of the FSM are said to be either accepting or not accepting. If when all input is processed the current state is an accepting state, the input is accepted otherwise not. Transducers generate output based on a given input and/or a state using actions and are typically used for control applications.

12. What can FSA do? Transducer FSA generate output based on a given input and/or state. In engineering transducers are used to describe a large number of systems: electronic design automation, communication protocol design etc. In biology and artificial intelligence research, state machines or hierarchies of state machines are sometimes used to describe, for example, cell operations and neurological systems In linguistics, state machines are sometimes used to describe the grammars of natural languages. 01/04/2014(c) Bishop: Consciousness and Computation12

13. 01/04/2014(c) Bishop: Consciousness and Computation13 The Turing Machine, (1935/6) A Turing Machine, (TM), consists of: A control box - a finite state sequential machine, [m]. A (potentially) infinite length tape, [T], divided into discrete frames with one symbol per frame. A r/w head which can read or write discrete symbols from a finite alphabet. The r/w head can move left or right unit frames along the tape.

14. 01/04/2014(c) Bishop: Consciousness and Computation14 Turing Machine operation The activity of a TM occurs in discrete steps defined by two conditions, (Q & I), to produce TM output (Z): Current internal state: q Current symbol on tape under machine head: s Produce output, [Z], { z 1 = symbol to write; z 2 = tape head movement } Given any current machine state Q, and an input symbol I, the state transition table specifies a 3-tuple, [z 1, z 2, Q}, defining the Turing machines operation: z 1 : Designates the symbol to leave on the tape. z 2 : Move machine r/w head left or right. Q: Designates the next Turing machine state.

15. 01/04/2014(c) Bishop: Consciousness and Computation15 An example Design a simple TMp to add two unary numbers. The numbers are zero terminated and also separated by a zero. Assume TMp is initial state Q1 with its r/w head over the leftmost [1]. Hence to add (2 + 3) the initial tape is [01101110] Symbol on tape =01 State: Q1{0 Q1 R}{0 Q2 R} State: Q2{1 Q3 R}{1 Q2 R} State: Q3{HALT}{HALT} In the Theory of Computation, the HALT state is sometimes refined into two sub- states: HALT-ACCEPT and HALT-REJECT. It is trivial to see that this does not extend the single HALT state TM model as accept and reject values can be encoded as alternative terminating states in the state transition table, (in the above example: Q3 and Q4).

16. 01/04/2014(c) Bishop: Consciousness and Computation16 The Universal Turing Machine Turing showed that in principle the operation of any Turing Machine, TM m (T), can be simulated by a Universal Turing Machine, UTM (m, T). Any TM m (T) can be encoded on a tape a series of ones and zero's, since every TM is described by its state transition table, (m), its program. The table (m) must be finite in length since the number of machine states and the input alphabet are both finite. The symbols on the tape of the UTM are divided into two groups: The left is the encoded description of TM m with a pointer indicating which state TM m would be in at this stage of the computation. The right is the sequence of symbols {X} that would be encountered by TM m on its input tape with a pointer indicating which square TM m would be currently scanning. Turing proved that the net effect of the universal machine, UTM (m, T), is exactly what the effect of Turing Machine, TM m (T), would be.

17. 01/04/2014(c) Bishop: Consciousness and Computation17 The Language of a Turing Machine The set of input strings that result in the TM M, entering an ACCEPT state is called the Language of M, and is denoted L (M). Turing Recognisable Any Turing Machine will always either {ACCEPT, REJECT or LOOP} when given any input string. LOOP occurs when a TM does not halt. LOOPing may occur due to the TM being caught in an infinite loop (it repeats the same steps forever) or be due to a more complex process. A language is Turing Recognisable if some Turing Machine ACCEPTs it. A Turing Recognisable Language is also called 'recursively enumerable'.

18. 01/04/2014(c) Bishop: Consciousness and Computation18 Deciders A Turing Machine that never LOOPs is called a Decider. ie. A Decider that is guaranteed to enter a HALT state on all possible input strings. A Turing Machine that is a Decider is also called 'defined Turing Decidable A language is Turing Decidable, if there exists a Defined TM that ACCEPTs it. A language that is Turing Decidable is also called 'recursive'.

19. 01/04/2014(c) Bishop: Consciousness and Computation19 Variants of Turing Machines A probabilistic Turing machine is one in which state transitions are defined by a probability distribution. A non-deterministic Turing machine is one which has more than one next state for some combinations of contents of the current cell and current state. An input is accepted if any move sequence leads to acceptance. Hence consider a nondeterministic Turing machine as a probabilistic Turing machine with all states leading to ACCEPT have transition probabilities one; or simultaneously unfolding as a computation tree, each branch of which follows one such transition possibility. State AB Q1 a R {0.7, 0.3)b R {0.9, 0.1) Q2 a R {0.5, 0.5)b R {0.5, 0.5) HALT-ACCEPT ## HALT-REJECT ##

20. 01/04/2014(c) Bishop: Consciousness and Computation20 Parallel processing (Multi-Tape Turing Machines) A multi-tape (parallel) Turing Machine (P) has k tapes and k read/write heads. It is trivial to show that the set of languages ACCEPTable by P is exactly the same as the set ACCEPTable by D. Similar to the UTM, encode each of the multi-tapes onto one tape and use pointers to indicate which tape to process next and where on the tape to process next.

21. 01/04/2014(c) Bishop: Consciousness and Computation21 Equivalence of ndTM and TMs A non-deterministic Turing Machine (ndTM) has an equivalent deterministic Turing Machine. Informal outline proof: We will show how to simulate any non-deterministic Turing Machine (N) by a deterministic Turing Machine (D). Consider the operation of N on input L. This defines a tree of possible execution traces through states of N. D must carefully evaluate all branches of N. As soon as D finds an ACCEPT state of N then D will also ACCEPT L, otherwise D will not terminate. It is clear that the language ACCEPTable by D is exactly the same as the language ACCEPTable by N.

22. Deterministic and non- deterministic automata Deterministic [and non-deterministic automata] recognise regular languages. A regular language is a language (i.e. set of strings) that can be expressed using a only simple regular expression often called a pattern. A pattern is simply a tool that specifies a set of strings via simple rules rather than by enumeration [via, say, a list]. The basic rules of a regular expression are: | vertical bar for OR Grouping symbols into a composite element via brackets () Quantification via ? means zero or one of the preceding elements; * means zero or more of the preceding elements; + means one or more of the preceding elements. Hence a simple regular language might be {Hot; Hat} as defined by the regular expression H(a|o)t 01/04/2014(c) Bishop: Consciousness and Computation22

23. Push-down automata and Turing machines Push-down automata recognise context-free languages. A context free language is generated by a pushdown automata (i.e. a simple automata with additional memory implemented via a stack). Hence a simple context-free language might be {ab aabb aaabbb aaaabbbb}. I.e. a set of all non-empty even-length strings, the entire first halves of which are a's, and the entire second halves of which are bs. Turing Machine (and non-deterministic Turing Machines) recogniserecursively-enumerable languages. Hence the relative power of automata is: [non] deterministic automata; push-down automata and (the most powerful) [non-deterministic] Turing machines. 01/04/2014(c) Bishop: Consciousness and Computation23

24. 01/04/2014(c) Bishop: Consciousness and Computation24 Turings thesis … Also known as the Church-Turing thesis: A Universal Turing Machine can perform whatever tasks a human computer can achieve when following an effective method by rote. And, (with only one or two exceptions), this thesis remains, agreed amongst logicians.

25. 01/04/2014(c) Bishop: Consciousness and Computation25 Other Universal Machines McCulloch/Pitts: crude neuron simulations Simulate OR, AND and NOT and hence build a computer. Kleene: Recursive Functions Foundation of functional programming. Church: Lambda calculus Foundation of LISP (McCarthy). Post Systems & Cholmsky Grammars (Character) String manipulation systems. NORMA The ultimate RISC machine...

26. 01/04/2014(c) Bishop: Consciousness and Computation26 And in the beginning … The first computers were hard-wired to perform specific functions. The British Colossus (1943) and the American ENIAC (1945) The stored program machine, derived from the UTM model, stores both data and program in one homogeneous memory. Re-programming the machine simply entails re-loading the memory. The 1 st Stored Program Machine, The Manchester baby, (1948) Commercialised by Ferranti (1951). The 2 nd, The Cambridge University EDSAC, (1949) LEO (1951) – sponsored by Lyons Coffee Houses… In America the UNIVAC (1951); The 1 st mass produced computing machine being the IBM 701 (1953) But no such [commercially available] stored program computer can out compute the UTM …