1. The Effect of Coding on Computation Adina Lederhendler Topics in Biological Physics 23/12/08 Shannon: A universal Turing machine with two internal states. Automata Studies, 1956 Miller: The magical number seven, plus or minus two: some limits on our capacity for processing information, Psychological Review, 1956

2. Outline What is a code? Abstract computation Different codes for universal Turing machines Real-world computation The use of coding in working memory

3. What is a code? Code – Set of rules for translation of input into output. Input 1 Input 2 Input 3 Input N I Output 1 Output 2 Output 3 Output N O

4. What is a code? Examples of codes: ASCII code English alphabet Binary digits Genetic code Nucleotide sequences Amino acids

5. Turing machine Information encoded as symbols on tape. Configuration: Symbol + Internal state. SiSi Reading/ writing head

6. Turing machine SiSi symbol + statesymbol + shift + state complexity of machine (size of code) Code ConfigurationStep

7. Universal Turing Machine SiSi Input tape and code of a Turing machine T UTM Same output as T. UTM not related to any specific computation.

8. Universal Turing machine Shannon, Automata Studies, 1956 Shannon, 1956 – what are the limitations on the code of a UTM? Minimal number of states for a UTM? Minimal number of symbols for a UTM? No 1-symbol UTM Cannot carry any information No 1-state UTM Each step depends only on the letter currently being read. Cannot carry enough information

9. How do alternative codes affect the computation? Universal Turing machine Universal Turing machine A SymbolsStates Symbols Universal Turing machine B States ?? Symbols Universal Turing machine C States ??

10. Two-state UTM Symbols Symbols that represent intermediate states during computation Symbols that represent input/output Universal Turing machine B States Information about states of A carried by symbols of B (symbol-state tradeoff)

11. Two-symbol UTM Universal Turing machine C States Information about symbols of A carried by states of C (symbol-state tradeoff) Symbols

12. A tape: C tape: Example – Machine A: m = 5, n Two-symbol UTM

13. Machine A Machine C

14. Two-symbol UTM Use information about machine A

15. Two-symbol UTM

17. Move to beginning of the next input Two-symbol UTM

18. Ready to begin next step. Two-symbol UTM

19. Symbol-state product CBA Symbols States Product Shannon: What is the minimum symbol-state product required to construct a universal Turing machine?

20. Minimal Turing machines Minsky, 1962 7-state, 4-symbol UTM. Later efforts Find additional minimal (size) UTMs. Find more efficient minimal UTMs. Woods, Theoretical Computer Science, 2008

21. Real world systems Best code for specific task Real-world considerations such as: Efficiency: How much time, what resources can we devote to the computation? Accuracy: How much noise is present in the system? How accurate does the computation need to be? Evolution/adaptation to specific task Biological systems

22. Working memory AMOUNT OF INFORMATION PER ITEM Miller, The Psychological Review, 1956 Miller, 1956 Attempt to quantify working memory capacity How many items can we recall immediately after being presented with a sequence?

23. Retrieval Adaptation through learning Sequences of items Recoding into “ chunks ” and storage in short-term memory Working memory New codes = More kinds of “chunks”

24. Working memory Baddeley & Hitch, 1974 PT V OBK All of B is included in A Some of C is included in B What relation must there be between A and C?

25. Summary A code is a set of rules for carrying out a computation. Different codes can be used to perform the same computation. The choice of code may affect the efficiency and accuracy of a computation. Biological systems can adapt codes to specific tasks.