Search and Access Strategies for Web Archives Sangchul Song and Joseph JaJa 3. Existing Access Methods 1. Background o The Web has become the main publication medium world- wide, covering almost every facet of human activity. However, the Web is an ephemeral medium. o Web archives offer unique opportunities for knowledge discovery due to the richness of their contents extending over significant periods of time. o We need effective and scalable access strategies for web archives covering significant temporal spans. 4. Problems With Existing Methods o Inefficient handling of time-constrained search. o Ineffective delivery of search results Inadequate relevancy scoring. Scoring is performed over the entire history. Ungrouped search results. URL is not unique in web archives – time dependent. Considering different versions of the same URL tend to have similar contents, It is highly likely to have the first result page “polluted’ by multiple versions of the same URL. Users can want to focus more on a specific time-period within the results. Lack of a group-scoring methodology. What group to show on the top is not clear without a group- scoring methodology. 2. Our Goals: Development of o An effective technology to explore and search for information in a web archive. The technology must enable effective information exploration and discovery. o A framework that produces ranking and evaluation of archived web contents within a temporal context as required by the user. o Methods to determine the relevance of a group of web objects within the temporal context. A group can consist of a series of temporally-contiguous versions of a single URL, or of web objects archived within some time span. o A framework that allows effective search using keywords and time spans for large scale web archives. 5. Overview of our Approach: o Efficient time-constrained search by maintaining separate inverted lists for a given time window  See Block 6. o Scoring within a temporal context by computing term weights as a function of time  See Block 7. o Grouping similar search results, while scoring search results as a group  See Block 9 and 10. Search all, and then Filter  Very inefficient!! September 11 attacks - Wikipedia, the free encyclopedia September 11 attacks - Wikipedia, the free encyclopedia The September 11 attacks (often referred to as nine-eleven, written 9/11) were a series of coordinated suicide attacks by al-Qaeda upon the United States on … en.wikipedia.org/wiki/September_11,_2001_attacks September 11 Digital Archive September 11 Digital Archive Uses electronic media to collect, preserve, and present the history of the September 11, 2001 attacks in New York, Virginia, and Pennsylvania and the public … 911digitalarchive.org/ 9/11 Tributes, September 11 Tributes and Memorials to the Victims … 9/11 Tributes, September 11 Tributes and Memorials to the Victims … Tributes of 9/11 - September 11th 9/11 memorials. For the Victims their Families and the many Heroes of September 11th /11 World Trade Center, k National Commission on Terrorist Attacks Upon the United States National Commission on Terrorist Attacks Upon the United States Commission chartered to prepare a full and complete account of the circumstances surrounding the September 11, 2001 terrorist attacks, k … and 4 million other pages pertaining to the September 11 th Attack … September 11 attacks - Wikipedia, the free encyclopedia September 11 attacks - Wikipedia, the free encyclopedia The September 11 attacks (often referred to as nine-eleven, written 9/11) were a series of coordinated suicide attacks by al-Qaeda upon the United States on … en.wikipedia.org/wiki/September_11,_2001_attacks September 11 Digital Archive September 11 Digital Archive Uses electronic media to collect, preserve, and present the history of the September 11, 2001 attacks in New York, Virginia, and Pennsylvania and the public … 911digitalarchive.org/ 9/11 Tributes, September 11 Tributes and Memorials to the Victims … 9/11 Tributes, September 11 Tributes and Memorials to the Victims … Tributes of 9/11 - September 11th 9/11 memorials. For the Victims their Families and the many Heroes of September 11th /11 World Trade Center, k National Commission on Terrorist Attacks Upon the United States National Commission on Terrorist Attacks Upon the United States Commission chartered to prepare a full and complete account of the circumstances surrounding the September 11, 2001 terrorist attacks, k … and 4 million other pages pertaining to the September 11 th Attack … Ethiopian calendar - Wikipedia, the free encyclopedia Ethiopian calendar - Wikipedia, the free encyclopedia Thus the first day of the Ethiopian year, 1 Mäskäräm, for years between 1901 and 2099 (inclusive), is usually September 11 (Gregorian),... en.wikipedia.org/wiki/Ethiopian_calendar - 43k APOD: September 11, Mars Global Surveyor: Aerobraking APOD: September 11, Mars Global Surveyor: Aerobraking September 11, 1997 See Explanation. Clicking on the picture will download the highest resolution version available. Mars Global Surveyor: Aerobraking … apod.nasa.gov/apod/ap html - 5k … and only 560 other pages that are irrelevant to the September 11 th Attack Ethiopian calendar - Wikipedia, the free encyclopedia Ethiopian calendar - Wikipedia, the free encyclopedia Thus the first day of the Ethiopian year, 1 Mäskäräm, for years between 1901 and 2099 (inclusive), is usually September 11 (Gregorian),... en.wikipedia.org/wiki/Ethiopian_calendar - 43k APOD: September 11, Mars Global Surveyor: Aerobraking APOD: September 11, Mars Global Surveyor: Aerobraking September 11, 1997 See Explanation. Clicking on the picture will download the highest resolution version available. Mars Global Surveyor: Aerobraking … apod.nasa.gov/apod/ap html - 5k … and only 560 other pages that are irrelevant to the September 11 th Attack “Find web pages that contain ‘September 11 th ’ before 2001” Chronological Listing Directory Hybrid Text-Search

6. Basic Techniques Determine a snapshot of web contents covering a time window SC k = { All web objects valid within a time interval [t k ~t k+1 ) } Given SCs, we maintain a set of inverted lists for each SC, and build a time hierarchy or a combined multi-version tree. 8. Search User Interface 9. Grouping Search Results 10. Group-wide Scoring Grouping is good, but now which group to place first on the result page? Simple method : use average or highest score among members More effective method: compute a relevancy score as a group. Instead of tf(t), we use df(t), document frequency of t in group. Instead of idf(t), we use igf(t), inverse group frequency. We extend some of the best known IR technologies for group ranking. 7. Scoring within a Temporal Context Relevancy scoring is based on the time that an web page was archived. The same contents will have different relevancy scores when the temporal contexts are different. (e.g. one was archived several months before the other) First page polluted by the same URL Grouped by URL (collapsed) Grouped by URL (collapsed) Grouped by Time Grouped by URL (expanded) Same contents, different archive dates  different scores!! SC 1 SC 2 SC K B- Tree PL SC1-w1 PL SC1-w2 w1w1 w1w1 w2w2 w2w2 PL SC2-w1 PL SC2-w2 w1w1 w1w1 w2w2 w2w2 PL SC1-wN wNwN wNwN B-Tree SC 1 SC 2 SC K SC 1 SC 2 SC K w1w1 w1w1 w2w2 w2w2 wNwN wNwN SC 1 SC 2 SC K PL SC1-w1 PL SC1-w2 PL SC2-w1 PL SCK-w1 Multi-version Tree Multi-version Tree PL SC1-w1 PL SC1-w2 w1w1 w1w1 w2w2 w2w2 wNwN wNwN w3w3 w3w3 PL SC2-w1 PL SC4-w1 PL SC1-w2 PL SC1-w3 PL SC1-wN Search and Access Strategies for Web Archives Sangchul Song and Joseph JaJa