1. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 1 Eye Movements and Eye Tracking Why move the eyes? see the same thing better (stabilize image, but not too well!) see something else (overcome low peripheral resolution) S. Anstis only small part of visual field is sampled at full resolution

2. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 2 Cortical Magnification in V1 Eric Schwartz Or in complex form: Approximately:

3. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 3 Types of eye movements Major Distinctions: conjugate vs. disjunctive abrupt vs. slow Types: vestibuocular reflex: counteract head rotation optokinetic reflex: counteract large field retinal motion smooth pursuit: counteract object motion (<30 deg/s) saccades: rapidly shift gaze direction (<600 deg/s) vergence movements: adjust vergence angle accomodation: adjust focus microsaccaes: counteract drift during fixation (1-2 Hz, <5’) microtremor:.5’, 30-100Hz

4. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 4 Saccades Yarbus (1950s) alternations of fixations and saccades, 3 per second

5. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 5 Yarbus, 1967: examine the picture at will estimate the economic level of the people estimate the people’s ages guess what the people were doing before the arrival of the visitor remember the people’s clothing remember the people’s (and objects’) position in the room estimate the time since the guest’s last visit

6. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 6 Saccade Metrics from Becker, 1991 approximately linear relationship between saccade amplitude A and saccade duration D: 20 ms < D 0 < 30 ms 2 ms/° < d < 3 ms/° Example: for 20° saccade display needs to be changed ~75ms after saccade onset

7. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 7 Anthropomorphic Robot Head

8. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 8 System overview Gigabit Router Desktop ADesktop BDesktop C

9. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 9 Motor System 9 DoF (hobby grade servo motors) 2 Mini SSC II interface boards Four bar linkage connection Eye/neck system is redundant

10. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 10 Vision System 2 Point Grey Research Firefly cameras (Firewire) 640x480 resolution up to 30 fps 2, 4, 6, and 8mm focal length lenses

11. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 11 Saccade Performance 1

12. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 12 Saccade Performance 2

13. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 13 Demo video

14. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 14 Why Eye tracking Basic Neuroscience: - measuring brain output - understanding neural control architecture - psychophysics: how do we use gaze during natural tasks Applications: - user interface design, human factors: e.g. eye tracking for driver of a car - advertising: do people look where I want them to look (in my web page, my newspaper ad, etc.) Note 1: now several conferences solely devoted to this Note 2: can also be done in fMRI Note 3: possible in the real world: “portable eye trackers”

15. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 15 Eye tracking techniques Eye tracking techniques (Review in Duchowski&Vertegaal, 2000) Contact lenses: with mirrors or induction loops - fast, accurate, uncomfortable (often used with bite bars) Video based: - temporal accuracy limited to camera frame rate (60Hz) - less accurate (~1 deg) - typically with infrared illumination of the eye (dark pupil vs. bright pupil) - can be done remotely or camera can be attached to head Electro-oculogram: - exploits electric dipole property of eye ball - fast but imprecise Limbus tracking: - predecessor of the video based tracker - imprecise - small operating range

16. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 16 Electro-oculogram eye is electric dipole electric field of moving dipole induces measurable voltages provides analog voltage output that can be digitized and processed extremely fast, but technique is not accurate electrode

17. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 17 Limbus Tracking utilize difference in reflective properties of iris vs. sclera infrared LED photo diode sclera limbus iris

18. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 18 Search Coil gold standard for speed and accuracy (5-10’’), but quite uncomfortable and head movement restrained

19. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 19 Video based tracking most widely used today, good accuracy and speed, easy-to-use dual purkinje tracker bright pupil image

20. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 20 inside of head mounted display U. of Rochester

21. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 21 Gaze contingent display changes Idea: decide what is being displayed contingent on where observer is looking Saccade contingent updating (SCU): a special case of this: make display changes while a saccade is progressing (pioneering work by McConkie and colleagues) powerful technique for studying visual perception frequently used in e.g. change blindness studies

22. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 22 Combination of video and limbus tracker inside HMD for gaze contingent display analog limbus tracker’s sensor with infrared LED, photo diodes (horizontal) video based pupil tracker’s sensor LCD displays inside the HMD video based tracker for good accuracy limbus tracker for low latency saccade detection

23. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 23 System Overview 4 processor high-end graphics computer as backbone images rendered in V8 helmet (Virtual Research) Three sensors: magnetic tracking device (Polhemus Fastrak) limbus tracker (ASL 210), sampled at 1.25 kHz pupil tracker (ASL 501), sampled at 60 Hz Sensors send data via separate serial lines

24. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 24 Latency Comparison in this example: limbus tracker has 26 ms advantage over video tracker on average: limbus tracker has 37±13 ms advantage (15 trials)

25. Jochen Triesch, UC San Diego, http://cogsci.ucsd.edu/~triesch 25 chess pieces (dis-)appear contingent on saccade (~25-30°) with 50% probability