1. Gaze-Augmented Think-Aloud as an Aid to Learning Sarah A. Vitak Scripps College Andrew T. Duchowski, Steve Ellis, Anand K. Gramopadhye Clemson University CHI 2012 5-10 May, Austin, TX John E. Ingram Sewanee: University of the South

2. Motivation: training Visual search: –well-defined strategies have been developed by experts –e.g., top-down cognitive strategies based on experience –Chest X-Ray (CXR) inspection: Airway, Bones, Cardiac silhouette, Diaphrams, External tissue,... Expert (left) and novice (right) scanpaths over abnormal CXR.

3. Objectives: histology Develop histological search strategy training Test effectiveness of expert’s gaze atop video –scanpath needs to demonstrate expert’s search strategy Task: identify BrdU marked cells in epithelial layer Example stimuli: immuno-gold (BrdU) stained cross-sections of bovine mammary tissue.

4. Previous work: tracking experts Ericsson et al. (2006) surveyed experts’ gaze: 1. experts’ search strategies are task dependent 2. experts’ shorter dwell times thought to reflect expertise 3. experts make better use of peripheral information 4. experts’ patterns of visual analysis develop with training 5. experts use a larger area around fixation 6. experts make better use of extra-foveal information Expert football player (Ronaldo, gaze captured with Dikablis tracker), expert pilots (Weibel et al., 2012 [ETRA]), expert laparoscopic surgeons (courtesy Stella Atkins and Bin Zheng)

5. Previous work: scanpath training Fertile research area (see paper for review) Selected references related to training: –static, stylized scanpaths (VR) Sadasivan et al. (2005) –dynamic scanpaths (still images) Nalangula et al. (2006) –static scanpaths (still images) Litchfield et al. (2010) –dynamic scanpaths (video) Jarodzka et al. (2010) Sadasivan et al. (2005). Jarodzka (2010).

6. Contributions: GATA Gaze-Augmented Think-Aloud (GATA) builds on: –Feed-forward training Sadasivan et al. (2005) hierarchical task analysis & scanpath visualization –Stimulated Retrospective Think-Aloud Guan et al. (2006) verbalizing while watching scanpath visualization –Eye Movement Modeling Examples (EMME) Jarodzka et al. (2010, 2009, 2010) task analysis & highlighting of salient regions (e.g., foveation)

7. Gaze-Augmented Think-Aloud GATA: –recorded scanpath & verbalization (audio track) –a specific visual search strategy is required –for histology images, devised STAMP: Staining, Tissue, Artifacts, Magnification, Plane of section Training video used in study.

8. Evaluation Experiment: –2 x 2 mixed factorial design: (presence or absence of video) x (experienced or naive participant) video between-subjects, 8 stimuli images within-subjects –task: “identify cells marked with BrdU in epithelial layer” –procedure: both groups saw training slides w/out scanpath Training slides: the first highlights marked cells, second differentiates tissue type, third shows epithelia with no marked cells, fourth again shows marked cells in the epithelia, fifth shows no marked cells; same verbalization was heard as in GATA video.

9. Evaluation Experiment: –apparatus: Tobii ET-1750 (see paper for specs) –participants: 32 (aged 19-57, median 22) 15 experienced, 15 naive 2 participants dropped (no data) –dependent measures speed (time to task completion) accuracy (hits & misses) fixation counts fixation durations –outliers: beyond 3 SD one removed (from A&VS) Example of participant & apparatus; half the participants (experienced) recruited from Animal & Veterinary Science (A&VS) class who were familiar with tissue images.

10. Dependent measures Speed –time to task completion –tabulated per each image, then averaged Accuracy –hits (true positives) –misses (false positives) Fixation counts Fixation durations –longer durations suggest cognitive load (e.g., difficulty) Example of two captured scanpaths: blue is from experimental group, red is from control group.

11. Results: speed Time to completion: –two-way ANOVA revealed significant effect of video F(1,6)=9.25, p<0.05) –but not of population group F(1,6)=0.01, p=0.92, n.s.) –those who saw video performed significantly faster in either group

12. Results: fixations Number of fixations: –two-way ANOVA revealed significant effect of video F(1,6)=8.93, p<0.05) –but not of population group F(1,6)=0.02, p=0.90, n.s.) –numbers of fixation closely mirror time to completion –not surprising as both are often correlated

13. Results: fixation durations Fixation durations: –two-way ANOVA revealed no significant effect of video F(1,6)=0.00, p=0.56, n.s.) –nor of population group F(1,6)=0.00, p=0.98, n.s.) –durations not expected to differ between groups –within-group analysis shows significant difference by naive group but not experienced group

14. Results: accuracy Considering only effect of video on accuracy: –one-way ANOVA revealed significant effect on error rate F(1,27)=10.15, p<0.01) –but not on correct responses F(1,28)=1.21, p=0.28, n.s.)

15. Discussion Results suggest fairly clear explanation: –those who viewed scanpath were faster with fewer errors –effect more pronounced in experienced group corroborates to a certain extent by Litchfield et al. (2010) Eye movements provide evidence of performance –numbers of fixation correlate with time might not be obvious had stopwatch been used –fixation duration may indicate expertise shorter fixations suggest faster recognition (familiarity with task)

16. Conclusion Gaze-Augmented Think-Aloud: –simpler than more elaborate visual codifications e.g., Sadasivan et al. (2005) –scanpath indicates what to look for and what to avoid –cost-benefit ratio is increased through simplicity –likely to be effective for CXR training Current training at local hospital: from left to right, intern, resident, radiologist.

17. Acknowledgments This work was supported by the US NSF –Research Experience for Undergraduates grant Thank you! Questions?