Advantages of Query Biased Summaries in Information Retrieval by A. Tombros and M. Sanderson Presenters: Omer Erdil Albayrak Bilge Koroglu

Outline Brief Introduction to Typical Information Retrieval System Why query biased summaries needed? How a query biased summary generated? Experimental Environment Experimental Results Conclusion and Future Works 2/13

Introduction 3/13 Typical IR System: input: information need output: ranked document list Figure 1. Screenshot from an IR System

Introduction (con’t...) Deciding whether the retrieved document is worth to further investigation Examining the static summaries Checking the whole content Weak content indicators: static summaries Time consuming job: refering full text nearly all times Aiming to minimise reaching whole content Generating query biased summaries 4/13

Generating Query Biased Summaries Query-specific summaries for each retrieved document Classical approach to summarization Extracting sentences Assigning scores to sentences Selection of best-scoring sentences Some modification on score assignment Extra importance to titles and subtitles More weights to sentences with clusters of terms Additional points to sentences includes query terms 5/13

Experimental Environment 6/13 TREC test collection: Wall Street Journal news 50 queries of which relevant documents are known 2 groups of 10 postgraduate students to find relevant docs for 50 queries One group with static summaries Another is to use query biased summaries 50 retrieved docs per each query 5 queries per student 5 minutes allocated per query Identical computers in hardware/software aspects

Experimental Environment (con’t...) 7/13 Figure 2. The subjects performing on query biased & static summaries in the experiments

Experimental Results 8/13 Recall: the ratio of total number of relevant documents for query to the number of retrieved relevant documents Precision: the ratio of total number of retrieved documents for query to the number of retrieved relevant documents Figure 4. Recall values of 2 groups Figure 5. Precision values of 2 groups

Experimental Results (con’t...) 9/13 Speed of evaluators’ judgments of relevancy 2.62% on 20 documents corresponds 13% increase of average number of examined documents Figure 6. The number of doc percentage for 2 group

Experimental Results (con’t...) 10/13 The need for checking full text Average number of full text reference per query: ‘query biased summary group’ : 0.3 ‘static summary group’: 4.74 Figure 7. Average number of references to the full text of documents per query

Conclusion and Future Works 11/13 Effective method: employing query biased summaries in IR Systems Easily identifiable more relevant documents Decreasing the need to check whole content of document Applicable to web search engines Expensive to retrieve the documenst from slow, not reliable, and remote servers Requiring to manage an index file More experiments on different summarization methods with different datasets

12/13 Thank you...