1. Eye Tracking and Interactive TV Interfaces ISE 298 Jean Ostrem

2. Eye Tracking Camera reflects IR beam on cornea and detects where the participant is looking. Two types: remote and head-mounted.

3. DIRECTV DVR

4. Research Questions Compare eye-tracking to think aloud protocol: Does eye-tracking provide different, more accurate information than the think aloud protocol? What happens if you use both simultaneously? Does thinking aloud skew the eye-tracking results?

5. Question 1: Dependent Variables Does eye-tracking provide different, more accurate information than think aloud protocol? Time on task Number of usability problems found Type of usability problems found Comparison of total mentions of an area of interest to number of times the area was viewed.

6. Question 2: Dependent Variables Does Think Aloud skew eye-tracking data if both are performed at once? Fixations (eye stationery for 100ms or longer) Total number per task type (Program Guide or Search). Total number per area of interest. Proportion of fixations in an area while it is visible. Average Fixation Duration Dwell time (cumulative time participants look at an area of interest). Total time per area of interest Total dwell time as a proportion of the time the area is visible.

7. DIRECTV Program Guide Record tonight’s Jeopardy. Record all episodes of The Colbert Report with no reruns Record all episodes of Numb3rs. Reruns are OK. Find out who is on David Letterman tonight and tomorrow night.

8. DIRECTV Search by Title Record the next episode of Mythbusters. Record all episodes of Damages with no reruns. Record all episodes of How I Met Your Mother. Reruns are OK. Find out if I Love Lucy is being broadcast. If so, what episodes are upcoming.

9. Test Design Think aloud protocol + eye tracking Silent + eye tracking Program Guide Tasks (n=4) Group A (n=5)Group B (n=5) Search Tasks (n=4)Group B (n=5)Group A (n=5)

10. SampleData Task: Record all episodes of The Colbert Report but make sure you don’t record reruns.

11. Question 1: Results Does eye-tracking provide different, more accurate information than think aloud protocol? Time on task: Think aloud protocol significantly longer for Search tasks only Number of usability problems found: No significant differences Type of usability problems found Eye tracking is better at finding failures to notice key areas of the screen Comparison of total mentions of an area of interest to number of times the area was viewed. Discussed later.

12. Question 2: Dependent Variables Does Think Aloud skew eye-tracking data if both are performed at once? Fixation Duration Sig. greater fixation durations when silent than when thinking aloud. Dwell time (cumulative time participants look at an area of interest). Data was too skewed to analyze. Fixations (eye stationery for 100ms or longer) Discussed later.

13. Areas of Interest Types Video Informational Interactive Instructional Title

14. Areas of Interest Types Instructional  Accurately reported during think aloud.  Sig. more fixations during think aloud.

15. Areas of Interest Types Interactive  Accurately reported.  Sig. more fixations during whichever protocol took longer (think aloud or silent).

16. Areas of Interest Types Informational  Inaccurately reported.  Sig. more fixations during silent tasks  Some indication of greater % of fixations during silent tasks.

17. Areas of Interest Types Video  Inaccurately reported during think aloud.  Sig. more fixations during think aloud

18. Areas of Interest Types  Too few fixations to analyze Title

19. Conclusions Eye tracking is better than the think aloud protocol for questions regarding how much attention participants pay to non-interactive and seldom accessed areas of the TV screen. It is more accurate at reporting gazes in these areas. It is better at finding failures to notice these areas. It is best not to use eye tracking concurrent with the think aloud protocol. Participants have more fixations in instructional areas because they read them out loud. Participants have less fixations in informational areas, possibly because they avoid reading them. Participants show somewhat less depth of processing because they are dividing their attention between manipulating the interface and talking to the moderator.