1. Results of Behavioral Data Procedures Tasks Objectives How Task Types and User Experiences Affect Information-Seeking Behavior on Web: Using Eye-tracking and Client-side Search Logs Hitomi SAITO (Aichi University of Education) Hitoshi TERAI (Tokyo Denki University) Yuka EGUSA (National Institute for Educational Policy Research) Masao TAKAKU (National Institute for Materials Science) Makiko MIWA (The Open University of Japan) Noriko KANDO (National Institute of Informatics URL: http://cres.jpn.org/ Email: cres@nii.ac.jp How do task types affect information-seeking behavior? ⇒ We compared informational and transactional tasks [1]. Informational task: report-writing Transactional task: trip-planning How do knowledge and experience affect search behavior? ⇒ We compared undergraduates with graduate students. [1] A. Broder. A taxonomy of web search. SIGIR Forum,36(2): 3–10, 2002. Summary of Results Windows XP Firefox Experimental data: Screen captured video Browser Logs Eye movements Eye-tracking system Lenses You are an undergraduate student and have to write a term paper for a world- history class. Your topic is ____________. You are an undergraduate student and have to write a term paper for a world- history class. Your topic is ____________. You plan to take a trip with ____________. The season you are traveling in is________ and you are going to _________ (location). You must give your trip partners such useful information as transit points, accommodation, locations, and events. You plan to take a trip with ____________. The season you are traveling in is________ and you are going to _________ (location). You must give your trip partners such useful information as transit points, accommodation, locations, and events. Number of Results Pages Queries Reading time for Results Pages Number of Non-results Pages Reading Time for Non-results Pages Actions Categories of Web search behaviors Participants Participants selected their own topics. Participants selected travel partners, season, and destination Report-writing TaskTrip-planning Task Screen-captured video Browser logs Manual annotations Eye movements 15 min. Pre-questionnaires How much do you use Web weekly? What browsers? What search engines? Pre-questionnaires How much do you use Web weekly? What browsers? What search engines? Post-questionnaire How difficult was task? Satisfied with search results? Post-questionnaire How difficult was task? Satisfied with search results? Interview Information-seeking processes Showed them screen-captured video Interview Information-seeking processes Showed them screen-captured video Search task Report-writing Trip-planning Repeat twice 15 min. ≒ 30 min. Search: searching with search engine Link: clicking on page link Next: going forward to next page Back: going backward to last page Jump: going to page in bookmark or history Browse: browsing next search results Submit: clicking submit button Bookmark: adding bookmarks Change: changing from one tab to another Close: closing a tab or window Report TaskTrip Task GraduatesUndergraduatesGraduatesUndergraduates MeanSDMeanSDMeanSDMeanSD Search9.203.358.004.637.805.896.365.18 Link25.8014.1319.186.6629.0014.4735.739.12 Next0.800.840.360.810.200.450.911.14 Back10.409.0717.187.9710.808.0422.0014.61 Jump2.201.922.361.693.202.682.451.75 Browse0.801.301.822.360.601.340.180.60 Submit7.6012.621.272.974.404.343.003.03 Bookmark8.001.414.552.168.006.444.552.42 Change43.6026.382.455.6328.4019.963.363.11 Close4.203.960.360.676.009.822.361.86 Location of blocks in lookzone Blocks Report TaskTrip Task GraduatesUndergraduatesGraduatesUndergraduates 1Title bar0.403.780.801.00 2Menu1.800.220.000.11 3Bookmark0.003.780.200.00 4Tool bar0.401.780.401.22 5URL bar0.400.780.000.11 6Search bar4.000.004.000.00 7Search bar button0.200.000.200.00 8Tab10.208.116.009.22 9Link for services2.4017.672.205.00 10Query box5.4036.893.0012.56 11Search button0.000.890.200.67 12Scroll bar0.600.110.00 13Number of hits0.000.440.600.00 14Sponsor Link0.006.6711.4012.11 15Spell check0.00 0.200.00 16Title38.8060.6739.2042.11 17Snippet70.0091.1128.4037.00 18URL16.6040.8912.4015.44 19Related search1.203.001.202.56 20Link for next page1.000.781.000.78 21Find in a page0.00 22Status bar0.001.780.00 Out of lookzone19.6052.8917.0018.22 Lack of eye position12.0083.447.2070.78 Eye-gaze points for each ranking Rank 1 Rank 2 Rank 3 Click rank data + Graduates Undergraduates User Differences Graduates: focused more attention on Search bar clicked higher ranked pages Undergraduates: focused more attention on tool bar, query box, and search button. clicked lower ranked pages User Differences Graduates: focused more attention on Search bar clicked higher ranked pages Undergraduates: focused more attention on tool bar, query box, and search button. clicked lower ranked pages Task Differences Report task: Both groups looked more at Scroll bar and Snippet Both groups tended to look and click lower ranked pages than trip task Trip task: Both groups looked more at Sponsor links Task Differences Report task: Both groups looked more at Scroll bar and Snippet Both groups tended to look and click lower ranked pages than trip task Trip task: Both groups looked more at Sponsor links User Differences Graduates more likely to: add bookmarks submit more forms change and close more tabs or windows Undergraduates more likely to: read Non-results pages crick on more links return to more previous pages User Differences Graduates more likely to: add bookmarks submit more forms change and close more tabs or windows Undergraduates more likely to: read Non-results pages crick on more links return to more previous pages 5 Graduates Library and information science majors Ages: between 23 and 28 Males: 4, female: 1 11 Undergraduates Various majors Ages: between 19 and 21 Males: 5, females: 6 Web browsers How do task types affect information-seeking behavior?  Task types affect user eye movements and clicking pages Trip task: Users looked more at sponsored links. Report task: Users looked more at snippets Users clicked and viewed lower rankings How do knowledge and experience affect search behavior?  User search patterns differed by experiences Graduates quickly searched in parallel by switching tabs or windows Undergraduates searched sequentially by alternating between links User and Task Differences Report task: Graduates browsed more Non-results pages Graduates performed more actions Task Differences Trip task: Undergraduates browsed more Non-results pages, and performed more actions than in report task User and Task Differences Report task: Graduates browsed more Non-results pages Graduates performed more actions Task Differences Trip task: Undergraduates browsed more Non-results pages, and performed more actions than in report task Instructions You have 15 minutes to collect related information through the Web. Add pages to browser’s book marks if useful. Results of Eye Movement Data ** + *