1. Situated Cognition: A New Direction in AI Bram van Heuveln

2. “Cognition is Reasoning” Traditionally, AI has seen reasoning as the ‘pinnacle’ of cognition: –Reasoning, problem-solving, decision-making, planning, is where the ‘real’ intelligence, and hence the ‘holy grail’ of AI, is considered to be. –Cognitive development and learning is deemed less interesting. –Sensory perception and motor control simply isn’t considered part of what AI people should study; these are often considered as ‘peripheral devices’ and ‘add- ons’ to the central reasoning system, just as the keyboard and monitor merely ‘relay’ or ‘transduce’ input and output.

3. The Traditional Model of Cognition: Sense, Plan, Act Thinking SensingActing Environment

4. Situated Cognition However, over the past 15 years or so, cognitive scientists have started to pay more attention to the body and the environment again. Situated (or Embodied) Cognition is the view that we have to take into account the body and the environment in trying to explain, and think about, cognition. Situated Cognition states that ‘Sensing’, ‘Thinking’, and ‘Acting are much more integrated (and not so cleanly separated) than initially believed, and that we shouldn’t be guided as much by the computer metaphor than we have been.

5. Catching a Fly Ball A standard example given by Situated Cognition people is the act of catching a fly ball. –The traditional solution to this problem would be to say that we perceive the trajectory of the ball (Sense), then calculate (Plan (compute!)) where the ball is going to be when, and how we need to act in order to get at that place at that time, and finally perform the sequence of actions that we just figured out would lead to the desired outcome. –However, on the situated view, you are much more interactive with your environment, and you follow a few simple rules: e.g. if in your field of vision the ball is moving left, you move left, and if it moves right, you move right: if you keep doing this, eventually the ball will come straight at you, so at least you’re in the path of the ball. In other words, the situated cognition people are claiming that certain cognitive acts do not involve complicated internal computations, but instead can (only!) be understood and explained by reference to the environment.

6. Top-Down Visual Processing Many people believe vision (or any other form of perception) to be a purely “bottom-up” process: starting with the raw sensory input, we recognize edges, bars, blobs, and other basic optical features. These features can then be combined (e.g. via the recognition-through-component approach) to recognize complex objects. There is neurological evidence for such processing, but there are also reasons to believe that the processing doesn’t only go from raw image to interpreted image, and that there are also “top-down” factors involved.

7. An Experiment in Vision

8. Active Perception It seems that contrary to popular belief, perception is not a passive kind of data- collection process, passing interpreted ‘snapshots’ on to higher-level processing. Rather, it seems that higher-level processes actively drive perception to a considerable extent, in that it ‘asks’ the visual system to look for certain things in the environment that the higher-level processes need.

9. The World As its own Best Model Situated cognition proponents like to point out that we don’t always form some kind of internal representation of the outside world Example: Blocks Experiment –In this experiment, subjects had to copy a certain configuration of blocks by selecting blocks and dragging them to the corresponding place. –The finding was that subjects would look at the original, then select a block, then look back at the original (!), and finally place the block. –On the traditional view of cognition, the third step would be a surprise. But, on the situated view, it makes sense.

10. Interaction as an Essential Part of Cognition Proponents of Situated Cognition claim that if you couldn’t interact with your environment, you wouldn’t have certain cognitive abilities! Some examples to support this view: –Tetris: People who play Tetris don’t look at the block, think about how they would have to rotate or move it, and then act on it. Rather, people rotate and move the blocks as they are trying to figure out where it fit. –Scrabble: People are finding words by moving the tiles around. If you would handcuff them, their cognitive ability to ‘think of’ words would decrease. –Long division: We use paper and pencil to do this!

11. The World as External Memory In fact, some Situated Cognition people say that the brain often uses the environment as a kind of ‘external memory’. Examples: –Suppose you are taking apart your computer (or something else with many parts) and want to make sure that you can put it together again. One strategy that many people use is to line up (or otherwise configure) the parts as they are taking them out, so that they just need to reverse this. –Any note you write to yourself to remember something. –In fact, planners, calendars, even laptops can be seen as your external memory.

12. Language: Our Best Tool? Language seems to be an especially powerful tool that we use to enhance our cognitive abilities: –Language can be used to represent information and thus serve as external memory (see examples before) –Expressions of language can be manipulated (logic, mathematics, science)

13. Example: Terminator http://www.youtube.com/watch?v=LUZgPfdkWis

14. The Extended Mind Continuing this line of reasoning, it would seem that the ‘boundaries’ of our cognitive self may well go beyond the boundaries of our biological self. Other examples: –If I have a hammer taped to my hand, then it’s more natural to draw the line between me (at least as a physically interacting being, but probably also as a being that perceives and cognitive interacts with its environment) and my environment at the tip of the hammer, rather than between my hand and the hammer (‘I hit the nail’). I become ‘hammer-man’! –Otto has amnesia, but uses a notebook to keep track of his experiences, appointments, etc. If Otto’s book is always-present and always-used, maybe it is better to consider the notebook as part of the cognitive entity we refer to as ‘Otto’ (why did Otto come to the appointment? Because Otto remembered it. Such a description of events would only make sense if the notebook is part of ‘Otto’

15. Is there only one ‘Cognitive Self’? Debates like the foregoing ask the question: where (or what) is my mind? Where is the boundary between my mind and its environment? These questions assume that there is one ‘cognitive self’, one cognitive system. However, what if we drop this assumption? Maybe there are various cognitive systems that one can point to, and that one can usefully refer to in order to give explanations and predictions of cognitive behavior. Note how this might play out quite nicely in some of the debates we’ve had: –The Chinese Room –Cognitive entities using language (science, math, tools) to form a new, more cognitively powerful, entity –The Otto case: ‘Otto1’ and ‘Otto2’