Commercial Uses of Eye tracking Workshop 5 th September 2005 – HCI Edinburgh This workshop was aimed at sharing and defining best practice, ideas, and unanswered questions on the scientific and practical use of eye tracking as applied to commercial UCD. Attendees: Linden Ball (Lancaster University, UK) Peter Brawn (Access Testing, Australia) Laura Cowen (IBM, UK) Stephen Gulliver and George Ghinea (Brunel University, UK) John McCarthy, Jens Riegelsberger (University College London, UK) Kara Pernice Coyne (Nielsen Norman Group, USA) Sharon O'Brien (Dublin City University, Ireland) Tony Renshaw (Leeds Metropolitan University, UK) Charlotte Sennersten (Gotland University, Sweden) Ekaterini Tzanidou (Open University, UK) Natalie Webb (Amberlight, UK) Alan Woolrych (University of Sunderland, UK) Daniela Zambarbieri (Università di Pavia, Italy)

1. Metrics and Definitions Fixations and scan paths within a known context (e.g. search) are accepted metrics for interpreting data Pupil dilation and blink rate however need more research to know the right conditions for their use; e.g. problems with luminance affecting pupil dilation Future needs Complete disclosure of study parameters; e.g. definitions of fixations. This will enable research comparisons. Discussions Scan path

2.Eye Mind Hypothesis Generally accept assumption that where people look indicates what they are paying attention to, or thinking about. However, care needs to be taken in respect of: The context of eye behaviour; e.g. a specific search task allows more confidence in inferences drawn whereas an open brief to look, means more factors are likely to influence behaviour, such as meaningfulness, visual (bottom-up) cues and motivational level Expectations, experience and individual differences will also influence behaviour The role of peripheral vision and pre-attentive processing cannot be directly determined by eye tracking and need to be inferred from eye movement data The hypothesis that users might consciously frustrate studies (e.g. changing task during an exercise) was considered possible but rare. Future work is required to establish the boundaries of the eye mind hypothesis for dynamic displays with variable frame rates e.g. video Discussions

3.Tools and Techniques Issue: How long does calibration hold good for and how does this influence the length of studies? The feeling was that more work is needed to test assumptions around the accuracy of calibration over time. 4. Study Design Quality of eye tracking studies depends on using proven study design principles; e.g. good sampling Ecological validity is important; e.g. helping users to relax and behave naturally in test conditions Eye tracking was seen to have an advantage in that the facilitator does not necessarily need to be in the same room as the participant Issue: How to predict sample size is still not clear due to lack of experimental data and universal definitions of eye tracking metrics. Future work: The group thought that there was potential for more studies on individual differences in eye movement; e.g. age, culture Discussions

5.Execution of studies The consensus was that participants should be informed of their eyes being tracked Though users of Tobii believed the equipment was fairly robust to natural movements during tests, all agreed that it was important to get the user in a sustainable position for the duration of the calibration and test Using re-play of eye movements to elicit feedback from participants post test was found to be useful but may carry the risk of post-hoc justification of actions by participants Think aloud protocol during eye tracking was seen to provide real-time feedback but altered behaviour. Post test replay was preferred by some Discussions

6.Analysis Care needs to be taken using averages; e.g. averaging different scan-paths may mean losing the ability to detect important differences. Issues: Analysis for dynamic data can be difficult; e.g. ‘Areas of Interest’ changing frame by frame for video Achieving synchronisation of eyetracker with multi-media stimulus; e.g. games Paper journals have limited effectiveness in showing analysis in more than 2 dimensions; e.g. changes over time. e-journals could provide a way to overcome this. Discussions

7.Validity of eyetracking Eyetracking is an attractive methodology for clients due to its novel and visual nature. Eyetracking could also be introduced more to clients as an additional analysis. Eyetracking is an excellent methodology for answering certain questions; e.g search strategies – it is real time and not dependent on self-report. It can also offer complementary evidence for much other work. Issues: Eyetracking is expensive due to equipment and additional analysis time in testing. Consequently, value behind additional expenditure needs to be thought through and explained clearly to the customer. Discussions