1. Evolution of Cognition Minds & Machines

2. Early Organisms: Perception and Action, but no Cognition SenseAct Environment Agent

3. Next Step: Cognition Think SenseAct Environment Agent

4. Next Step: More (Better) Cognition! Think! SenseAct Environment Agent

5. “Cognition is Thinking” Traditionally, Cognitive Science and AI have seen thinking as the ‘pinnacle’ of cognition: –Thinking includes reasoning, problem-solving, decision-making, and planning –Sensory perception and motor control is much less considered, if at all. These are often considered as ‘peripheral devices’ or ‘add- ons’ to the central reasoning system, just as the keyboard and monitor merely ‘relay’ or ‘transduce’ input and output.

6. Devils, Angels, Aliens, Jesus, Mary, and … Illinois! http://jesuspan.com/ Jesus Mary

7. An Experiment in Vision

8. Moral: Perception and ‘Cognition’ are integrated It seems that contrary to popular belief, perception is not a passive kind of data- collection process, passing interpreted ‘snapshots’ on to higher-level ‘cognitive’ processing. Rather, high-level processes (such as background knowledge or desires) affect how we perceive things (perception is constructive), and often ‘ask’ the visual system to look for certain things in the environment that are needed for the task at hand (perception is selective).

9. Situated Cognition Situated (or Embodied) Cognition is the view that we have to take into account the body and the environment in trying to explain, and think about, cognition. Situated cognition objects to the classical ‘Sense, Plan, Act’ model of cognition, which many cognitive scientists, most AI researchers, have assumed in their view of cognition: –we perceive the world, –then think about it (i.e. cognize about it!), –and then act on whatever plan we came up with. Instead, Situated Cognition proponents say, perception and action are integral to cognition.

10. Catching a Fly Ball How do we catch a fly ball in the outfield? –The traditional solution to this problem would be to say that we perceive the trajectory of the ball (Sense), then calculate (Think) where the ball is going to be when, and finally move to the right place (Act). –However, on the situated view, you are much more interactive with your environment, and you follow a few simple rules: e.g. if in your field of vision the ball is moving left, you move left, and if it moves right, you move right: if you keep doing this, eventually the ball will come straight at you, so at least you’re in the path of the ball.

11. The World As its own Best Model Situated cognition proponents like to point out that we don’t always form some kind of internal representation of the outside world Example: Blocks Experiment –In this experiment, subjects had to copy a certain configuration of blocks by selecting blocks and dragging them to the corresponding place. –The finding was that subjects would look at the original, then select a block, then look back at the original, and finally place the block. –On the traditional view of cognition, the third step would be a surprise. But, on the situated view, it makes sense.

12. Copying Blocks Experiment OriginalCopy Bins Task: Subjects have to make a copy of the configuration of blocks on the left by ‘grabbing’ individual blocks from the bins at the bottom and placing them on the right

13. Some Other Examples Tetris: –People who play Tetris don’t look at the block, think about how they would have to rotate or move it, and then act on it. Rather, people rotate and move the blocks as they are trying to figure out where it fit. Scrabble: –People are finding words by moving the tiles around. If you would handcuff them, their cognitive ability to ‘think of’ words would decrease. Long division: –We use paper and pencil to do this!

14. The World as External Memory Situated Cognition people say that the brain often uses the environment as a kind of ‘external memory’. Examples: –Suppose you are taking apart your computer (or something else with many parts) and want to make sure that you can put it together again. One strategy that many people use is to line up (or otherwise configure) the parts as they are taking them out, so that they just need to reverse this. –Any note that you often write to yourself to remember something. –In fact, planners, calendars, even laptops can be seen as your external memory.

15. Interaction as an Essential Part of Cognition Based on these examples, some proponents argue that, at least in some situations, our cognitive ability is not a function of our brain alone, but is also due to interactions with our environment.

16. Tools to Enhance Cognition To become smarter, then, maybe we don’t need bigger brains, but better interactions with our environment. Indeed, we can see the use of tools as a straightforward example of enhancing our abilities, so why not have cognitive tools?

17. Cognitive Ergonomics Designing tools with an eye for our cognitive abilities (and hence the constraints it imposes on that technology): cognitive ergonomically designed tools should ‘fit’ our abilities to take in and process information, and act on it.

18. Tuesday, Nov. 2, 3pm, EMPAC Presidential Lecture! Craig Mundie, Chief Research and Strategy, Microsoft “New trends in technology are transforming how we interact with computers. … more natural experiences using voice, touch and gestures …wall-sized displays … will enable the next generation of immersive computing experiences. Microsoft’s Craig Mundie will discuss these trends and show some of the possibilities of human- scale interaction”

19. The Extended Mind Can the ‘boundaries’ of our cognitive self go beyond the boundaries of our biological self? If you would be willing to include a brain chip implant to be able to see again as part of the cognitive being you’re dealing with, why should it matter whether this chip is actually inside our skull, or is in contact with the rest of our brain through radio-transmitter?

20. Hammer-Man Suppose I have a hammer taped to my hand that never comes off. Maybe it is more natural to draw the line between me and my environment at the tip of the hammer, rather than between my hand and the hammer. As always in science, we want to ‘parse’ the situation as naturally as possible so as to most leverage our explanatory and predictive powers. I myself would, just as with the sensory substitution cases, probably start perceiving the world with the hammer as part of me, not of my environment. Likewise, you would probably view me as ‘hammer-man’!

21. Blind-Cane-man Bob is blind, and uses a cane to feel around. Is the cane part of Bob? It isn’t part of Bob as a biological being. But is it part of Bob as a cognitive being? –There is a cognitive agent here, perceiving the world, thinking about the world, etc. –Is the cane part of the cognitive agent, or part of the world? Is Bob a cyborg?

22. The Curious Case of Otto Otto has amnesia, but uses a notebook to keep track of his experiences, appointments, etc. If Otto’s book is always-present and always-used, maybe it is better to consider the notebook as part of the cognitive entity we refer to as ‘Otto’ “Why did Otto come to the appointment? Because Otto remembered it.” Such a description would only make sense if the notebook is considered part of ‘Otto’.

23. Extended Minds? Selfs? Consciousnesses? Does the extended mind hypothesis imply an extended self? Extended consciousness?!?

24. Kevin Warwick, Cyborg http://singularityhub.com/2010/03/09/kevin -warwick-once-a-cyborg-now-a-prophet-of- the-man-machine-future-video/#more- 13202http://singularityhub.com/2010/03/09/kevin -warwick-once-a-cyborg-now-a-prophet-of- the-man-machine-future-video/#more- 13202

25. Is there only one ‘Cognitive System’ per Biological Body? The question “Where is the boundary between my mind and its environment?” assumes that there is one cognitive system to be defined. However, what if we drop this assumption? Maybe there are various cognitive systems that one can point to, and that one can usefully refer to in order to give explanations and predictions of cognitive behavior. So, in the Otto case, maybe we can (and should) meaningfully distinguish between ‘Otto1’ and ‘Otto2’. Some explanations work better by reference to Otto1, and other explanations work better by reference to Otto2.

26. Does Tool Use create a new Cognitive System? Cognitive System A World Cognitive System A Tool World Cognitive System B

27. Language: Our Best Tool? Language seems to be an especially powerful tool that we use to enhance our cognitive abilities: –Expressions of language can be used to represent information and thus serve as external memory (see examples before) –Expressions of language can be manipulated and thus reasoning and decision-making can take place (logic, mathematics, science)

28. Objection: Higher-Order Cognition without Interactions OK, but then what about someone who is just sitting in a chair, contemplating something or other? And what about Stephen Hawking, the world-famous physicist who is in a wheelchair and whose motor neuron disease has left him with hardly any motor skills left?

29. A Possible Answer Interestingly, even in the case of Stephen Hawking, situated cognition could argue that his (first-class!) high-level reasoning abilities depend on lower-level abilities: –Maybe the ‘thinking’ or ‘reasoning’ that we do (including Stephen Hawking) is a kind of ‘internalized dialogue’: an imagined dialogue in which we discuss (using language of course) something or other with some one else, but where we have taken the role of this other person, and where the brain has simply made a short-cut between the motor cortex straight to the perception cortex. –Thus, to explain the high-level cognitive powers of Stephen Hawking, we still make reference to his low- level perception and action abilities: although they are going on inside in his brain.

30. ‘Internal’ Representations as ‘External’ Environments This idea can be generalized: parts of our brain can be used to represent the world, but these representations are what other (more ‘core’, cognition-wise) parts of the brain interact with in order to accomplish cognitive tasks. This does suggest that many internal representations are ‘lower-level’, i.e. sensory- motor sequences rather than abstract symbols.

31. Internal Representations Cognitive System A World Cognitive System A World Cognitive System B

32. Example: Terminator http://www.youtube.com/watch?v=LUZgPfdkWis http://www.youtube.com/watch?v=Lf31v0eiXQk

33. The Chinese Room Remember the Chinese Room Objection: Systems Reply: the person inside the room does not understand Chinese, but maybe the larger system, of which the person is only a part, does. Counter-Objection: OK, have the man memorize the whole rule book, and do all the symbol manipulations in his head. Now there is no larger system to point to. Counter-Counter-Objection: Yes there is! Part of the brain of the man is now representing something, and other (original) parts of the man’s brain are interacting with it. Together, that forms a new (larger) cognitive system.

34. Still: Is computation just a tool? Cognitive System A World Cognitive System A Computation World Cognitive System B

35. Syntactic Computation and Semantic Computation We have 2 notions of computation! –Syntactic Computation: Going through a process of symbol manipulation (which can be mechanized: computer as we know it). –Semantic computation: Representing something by some symbol string, manipulating that symbol string into a new symbol string (possibly using syntactic computation), and interpreting that new symbol string into something meaningful. Who is doing the computation?!?

36. Syntactic and Semantic Computation xy I O f M Syntactic Computation Semantic Computation

37. Multiple Minds? Selfs? Consciousnesses? If we analyze a human in terms of multiple cognitive systems, does that mean that there are also multiple minds? Multiple selfs?? Multiple consciousnesses?!?

38. Language, Tools, Culture, and Cognition Whatever your stance on extended cognition or the self is, this much is clear: Language and other kinds of tools seem to greatly expand our cognitive capabilities. Moreover, language allowed us to write down thoughts and skills, and not only pass them to others around us, but also to people born long after we are dead: culture!

39. Cultural Evolution of Cognition Once culture is in place, cultural evolution can work on components of cognition. For example, such cognitive ‘building blocks’ as concepts or ideas, but also fashions and values can be passed from organism to organism, where they ‘compete’ for existence (certain ideas strike us as better ones than others’, and can get mutated or combined with others: all the ingredients that an evolutionary process requires. Richard Dawkins coined the term ‘memes’ for these kinds of entities that are subject to cultural evolution.

40. Application of the Idea of Memes The history of science and mathematics is full of cases where two or more researchers come up with the same idea or result, but independently so. How strange!! However, there is probably a good reason for this, and that is that the concepts and ideas (the memes) they needed to put together in order to obtain the theory of evolution had been developing over time, and were ‘ready’ to be combined at their time, i.e. the discovery was ‘in the air’. So, if they wouldn’t have come up with it, someone else probably would have very soon after. Note that this makes it rather arbitrary to credit 1 particular person with a particular idea. More likely, ideas are the natural progression of the work of hundreds of people preceding the person that ‘puts it all together’.

41. Wallace’s Paradox Alfred Wallace (a contemporary of Charles Darwin who came up with the theory of evolution independently of Darwin!) posed the following puzzle: –Our ancestors from, say, 100,000 years ago had almost equally big brains (and almost equal genes) than we have, but basically all they did was hunt for food: they did no abstract math, build spaceships, write PhD’s, etc, even though they presumably could (e.g. if you had a time-machine and were to transport them to our time, they would probably be doing math, and writing PhD’s, just like us). –So what did they need those big brains for? More to the point: why would evolution push for big brains if they are not used to the potential that they apparently can be used for? This seems a waste of resources and energy that seems to go against basic evolutionary principles: a paradox!

42. One Possible Answer to Wallace’s Paradox One possible answer to Wallace’s Paradox is to say that the hunting and gathering actually requires a lot more cognitive power (e.g. sophisticated perception and action and social cooperation) than you might think. Thus, the implicit assumption in Wallace’s reasoning is the good old intuition that big brain = big cognition = reasoning = mathematics, rockets, and PhD’s This may well be a mistaken view on cognition (e.g. about a full 1/3 of our brain is devoted to just vision!)

43. Another Answer to Wallace’s Paradox Another assumption implicit in Wallace’s paradox is the view that our cognitive capacities are the result of our brain alone. Under this assumption, our cognitive capacities would be purely the result of genetic factors, and biological (i.e. genetic) selection may indeed not be able to explain the drastic increase in cognitive powers. However, as argued in previous slides, our cognitive capacities may be partly derived form the artefacts (tools, language, and in particular science and math) we have created around us and pass along from one generation to the next. Moreover, the cultural evolution of cognitive building blocks such as concepts and ideas is *much* faster than genetic evolution.

44. Evolution and ‘Perfect’ Cognition Where is evolution going to bring us? Cyborgs? The ‘Singularity’? Consider the “Perfect Cognizer”, which is a hypothetical organism with ‘perfect’ perception, ‘perfect’ memory, ‘perfect’ reasoning, etc. Thus, the Perfect Cognizer takes in all factors relevant to a certain situation, and tries to figure out exactly which course of action is the most advantageous, so one would think that the Perfect Cognizer would do really well as an organism. Maybe evolution will bring us ‘The Perfect Cognizer’? Problem: Organisms must often make quick decisions in order to survive. The Perfect Cognizer probably takes too long to come to quick decisions! So, it is unlikely that we would evolve into perfect cognizers, let alone that we are perfect cognizers. Indeed, the following slide provides some evidence that humans are not perfectly rational or logical. It is exactly these kinds of imperfections to the human mind (just like imperfections to the human body (the blind spot, the appendix, the tailbone, the tonsils, etc)) that point to our evolutionary history.