1. Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness Roger Johansson, Jana Holsanova, Kenneth Holmqvist Department of Cognitive Science, Lund University, Sweden Eye movements as a window to the mind Jörg Brunstein

3. Theoretical background Mental images: Do they exist? And if yes, how do they work? Relevance: Where do people look when reflecting, remembering the last display etc.?

4. Hypotheses Similar EM when seeing pictures, remembering pictures, and retelling verbal descriptions. (a)EM indicate locations of objects (b)Equally strong for retelling and remembering (c)In light and in complete darkness

5. Experiments 1.Listen and retell in light 2.Look and retell in light 3.Listen and retell in complete darkness 4.Look (in light) and retell in complete darkness Data: EM during encoding (see/listen) and retrieval (describe/retell)

6. Methods 12 + 12 + 28 + 28 participants (50% (fe)male) SMI iView X at 50 Hz (bicycle helmet) –Glasses and lenses no problem, but mascara White board 657 x 960 mm (about 36 x 38 inches) in 150 cm distance (about 59 inches)

7. Welcome to our lab!

8. Calibration

9. Phase 2 Look: “You will soon see a picture. We want you to study the picture as thoroughly as possible. While you study the picture we will measure you pupil size.” (30 sec.) Listen: “Imagine a two-dimensional picture. There is…” (2 min. 6 sec.)

10. Look!

11. Retell!

12. Interview 1.What do you think the objective of this study was? 2.Rate the vividness of your visualization during the description phase on a scale ranging from 1(not very vivid) to 5 (extremely vivid). 3.Access whether you usually think in pictures or words.

13. Data

14. Experiments 1 - 4 Experiment 1: Listen and retell in light (2 min + 1-2 min) Experiment 2: Look and describe in light (30 sec + 1-2 min) Experiment 3: Listen and retell in complete darkness (2 min + 1-2 min) Experiment 4: Look (in light) and describe (in complete darkness) (30 sec + 1-2 min)

15. Data analysis Temporal reference: Holsanova (2001): eye-voice latencies typically between 2 and 4 sec. –listening: looking after hearing (2.1 sec) –retelling: looking before or after telling (0.29 sec.)  5 sec. before and after onset

16. Data analysis Spatial reference: Global correspondence: looking to correct position relative to complete scene (direction + distance) (up, down, left, right; full/half distance; stand still) Local correspondence: looking into correct direction No correspondence: neither location nor direction within time window

17. Results Exp.LocalGlobal Listen (L)64.354.8 Retell (L)74.955.2 Describe (L)74.854.4 Listen (D)60.827.6 Retell (D)64.935.1 Describe (D)69.040.5

18. Discussion: Content Eye movements reflect positions of objects Retelling = describing from memory Functional role of eye movements for mental images as spatial indexes (working memory, simulated vision, utterance planning)

19. Discussion: Methods Complex pictures and images Spatial arrangements instead directions only Relative ROIs instead of absolute ones for scaling effects Mental images instead of visual percepts Tracking in light and in complete darkness Minor: don’t say that they track directions but calibrate participants