La Technologie des Mouvements Oculaires en Linguistique Expérimentale Rachel Shen

The key points of this presentation What is eye tracking? Eye tracking in language studies and some examples; A very brief evaluation of different types of eye trackers; The SMI eye tracking glasses

What is eye tracking? Subconscious movements of the eyes during visual activities, such as scene viewing, reading, etc.; Subconscious movements of the eyes during visual activities, such as scene viewing, reading, etc.; Intake of information through those movements will be processed by human brain; Intake of information through those movements will be processed by human brain; The eye-mind assumption and the lag between attention and fixations; The eye-mind assumption and the lag between attention and fixations; Track them! Track them!

Eye tracking in Language studies: READING Self-paced reading; Gaze contingency paradigm; Linguistic manipulations can be reflected in the measurements of “fixation time” and “saccade time and direction” From a study of speed reading made by Humanist laboratoriet, Lund University, in Data are recorded using an SMI iView X 240 Hz video-based pupil-corneal reflex eye tracker.

Reading example: Normal sentence

Reading example: Syntactic anomaly

Reading example: Semantic anomaly

Eye tracking in Language studies: Visual World Paradigm (VWP) Auditory input and visual input; Eye movements (most commonly, no. of fixations and saccades) on the visual input are recorded; Sensitive to various linguistic properties from word recognition to syntactic ambiguity resolution (and beyond)

VWP: the Advantages Compared to Reading Provides clues about what happens prior to a critical word or region (can measure attention); Measures relative direct interpretations of a listener, which might be misjudged by the listener himself; Some phenomena can only be measured in situated condition; Easier to apply with children and patients

Example VWP: Donald Duck and tractor (Conklin, Dijkstra & van Heuven, 2008) HE ms ms

Example VWP: Cue integration during spoken word recognition (Toscano & McMurray (2012) VOT (400ms) vs. VL (700ms) VOT time course

Eye tracking in Language studies: Production No spoken input; To name an object or describe a scene; Gaze movements and object viewing order are recorded, especially relative to the speech output; Reflects utterance planning

Example Production (Gleitman, January, Nappa & Trueswell, 2007) Onset of each stimuli were given to manipulate the attention; Fixation order is affected by the manipulation; Word order choice for utterance production is affected;

The integrated paradigms Situation specific; Pragmatic inferencing; Dialogue

Dialogue The map task (Anderson et al., 1991); Tree decoration task (Ito and Speer, 2006); Tangram task (Clark and Wilkes-Gibbs, 1986); Lego task (Clark and Krych, 2003); Baufix ( Poncin & Rieser (2006); Cooking (Hanna & Tanenhaus, 2004)

Example Situated: cue encoding and decoding ( ) Example Situated: cue encoding and decoding (Snedeker & Trueswell (2003)) Tap the frog with the flower. (Ambiguous) Tap the frog by using the flower. (Unambiguous) Tap the frog that has the flower. (Unambiguous)

Eye tracking Data presentation Animated; Static saccade path; Heat maps; Blind zone maps US viewers Canadian viewers

Which eye tracker? Setups vary; Setups vary; Sample rates vary (30Hz-2000Hz); Sample rates vary (30Hz-2000Hz); Bright pupil (varying light conditions) and dark pupil tracking (Lab condition); Bright pupil (varying light conditions) and dark pupil tracking (Lab condition); For detailed comparison: For detailed comparison:

The SMI Eye tracking Glasses System Type: Video based glasses-type eye tracker Sampling Rate: 60Hz binocular Method: Dark pupil, pupil-cr Binocular Tracking: Yes (auto parallax correct) Accuracy: 0.5 degrees over all distances Gaze tracking range: 80° horizontal, 60° vertical Additional Details: HD Scene Camera Resolution: 1280x960

The workflow of the SMI Eye tracking Glasses Design Record Analyse

The SMI Eye tracking Glasses Example

Field work with the SMI Eye tracking Glasses Easy to carry (1300g /246g); Easy to carry (1300g /246g); Easy to setup; Easy to setup; Easy to calibrate and validate with live feedback on scene video; Easy to calibrate and validate with live feedback on scene video; Long recording time (2-4hrs); Long recording time (2-4hrs);

Data view and analyses with the SMI Eye tracking Glasses Event replay such as gaze size, scanpath, etc.; Event replay such as gaze size, scanpath, etc.; Raw data of fixations and saccades in terms of location, start time and end time, etc. Raw data of fixations and saccades in terms of location, start time and end time, etc.

Please come and try it! Merci beaucoup pour votre temps!