Tag Data and Personalized Information Retrieval 1

Abstract This work investigate the use of tag data for evaluating personalized retrieval systems involving thousands of users. They demonstrate how one can rate the quality of personalized retrieval results. They show that a user's “bookmark history" can be used to improve search results via personalization. 2

Introduction (1/4) Researchers require personalized relevance judgments to evaluate their systems. ▫ Documents are deemed relevant for a particular query by a particular individual. An excellent source of such information is personal query logs and click-through data, Z. Dou (2007). Query logs are not readily available to the wider research community due primarily to privacy and monetary concerns. The standard test collection in IR, namely the TREC datasets, cannot be used for evaluating personalized IR systems. ▫ The topics (queries) and corresponding relevance judgments are not associated with particular users. One public source of personalized ratings is tag data. 3

Introduction (2/4) Tag Data Social bookmarking systems such as del.icio.us, StumbleUpon and Bibsonomy are a recent and popular phenomenon. In these systems, users label interesting web pages with tags. These sites oer an alternative model for discovering information online. ▫ Users can browse tags for popular pages that have been tagged by a number of different users. These systems can be seen to provide consensus categorizations of interesting websites. 4

Introduction (3/4) Various researchers have investigated the applicability of social bookmarking data to improve Web search results, S. Bao (2007), Y. Yanbe (2007). Heymann (2008) found that the bookmark data had a good coverage of interesting pages on the Web. ▫ Bookmarked URLs were disproportionately common in search results given the small relative size of the del.icio.us index. ▫ Over a set of 30,000 popular queries, they found that 19% of the top 10 results. ▫ 9% of the top 100 results were present in the index. 5

Introduction (4/4) Whether social bookmarking data can be used to improve Web search from the perspective of personalization? ▫ Can tag data be used to approximate actual user queries to a search engine? ▫ How can we evaluate personalized IR systems using information contained in social bookmarks (tag data)? ▫ Is there enough information in the tags/bookmarks in a user's history in order to build a profile of the user that will be useful for personalizing search engine results? 6

Related Work 7