Presentation is loading. Please wait.

Presentation is loading. Please wait.

Archival HTTP Redirection Retrieval Policies Temporal Web Analytics Workshop 2013, Rio De Janiro Ahmed AlSum, Michael L. Nelson Old Dominion University.

Published byEdmund Buck Matthews Modified over 8 years ago

Similar presentations

Presentation on theme: "Archival HTTP Redirection Retrieval Policies Temporal Web Analytics Workshop 2013, Rio De Janiro Ahmed AlSum, Michael L. Nelson Old Dominion University."— Presentation transcript:

1 Archival HTTP Redirection Retrieval Policies Temporal Web Analytics Workshop 2013, Rio De Janiro Ahmed AlSum, Michael L. Nelson Old Dominion University Norfolk VA, USA {aalsum,mln}@cs.odu.edu Robert Sanderson, Herbert Van de Sompel Los Alamos National Laboratory Los Alamos NM, USA {rsanderson,herbertv}@lanl.gov

2 Agenda Introduction Abstract Model Experiment And Results Retrieval Policies

3 Memento Terminologies URI-R, R URI-M, M URI-T, TM http://www.amazon.com http://web.archive.org/web/20110411070244/http://amazon.com Original Resource Memento TimeMap

4 Live Redirect http://bit.ly/r9kIfChttp://bit.ly/r9kIfC redirects to http://www.cs.odu.eduhttp://www.cs.odu.edu % curl -I http://bit.ly/r9kIfC HTTP/1.1 301 Moved …. Location: http://www.cs.odu.edu/ …

5 Live Redirect http://bit.ly/r9kIfChttp://bit.ly/r9kIfC redirects to http://www.cs.odu.eduhttp://www.cs.odu.edu

6 Archived Redirect www.draculathemusical.co.uk redirects www.dracula-uk.com/index.html http://api.wayback.archive.org/memento/20020212194020/http://www.draculathemusical.co.uk/ redirects http://api.wayback.archive.org/memento/20020212194020/http://www.geocities.com/draculathemusical

7 Abstract Model

8 M1M1M1M1 M1M1M1M1 M2M2M2M2 M2M2M2M2 M3M3M3M3 M3M3M3M3

9 URI Stability URI’s stability is a count of the change in HTTP responses across time (200, 3xx, or 4xx) and the number of different URIs in the “Location” for 3xx status code. High Stability = 1 No Stability = 0

10 Timemap Redirection Categories All Mementos have 200 HTTP status code All Mementos have redirection to the same URI. All Mementos have redirection to different URIs. Mementos have different HTTP status code. Stability =1 Stability ≈ 0

11 URI Reliability M1M1M1M1 M1M1M1M1 3xx3xx M2M2M2M2 M2M2M2M2 3xx3xx M3M3M3M3 M3M3M3M3 3xx3xx rel=original R` MM rel=original R` MM rel=original R` MM Stability =1 ?? ?? ?? 200200 404404 3xx3xx

12 HTTP Redirection Relationship between URI-R & URI-M Case 1 Case 2Case 3Case 4Case 5

13 Experiment & Results

14 Experiment Dataset: 10,000 sample URIs from HTTP Status/Code Percentage (10,000 URI-R) OK (200) 82.83% Redirection (3xx) 14.71% Redirection (301) 8.4% Redirection (302) 6.1% Redirection (others) 0.2% Not-Found (4xx) 1.18% Others 1.28% HTTP Status/Code Percentage (894,717 URI-M) OK (200)93.46% Redirection (3xx)5.69% Not-Found (4xx)0.26% Others0.59% URIs Live HTTP status code Memento HTTP status code

15 Time spanNumber of Mementos

16 URI Stability Stability in semi-log scale Stability for |TM(R)| < 300

17 URI Reliability Reliabilityin semi-log scale Reliabilityfor |TM(R)| < 300

18 HTTP Redirection Relationship between URI-R & URI-M Case 1 Case 2Case 3Case 4Case 5 80.8% 2.74% 1.34% 1.33% 13.7%

19 RETRIEVAL POLICIES ARCHIVED HTTP REDIRECTION RETRIEVAL POLICIES

20 Current Wayback Machine Policy

21 Policy one: URI-R with HTTP redirection Retrieve the memento M for R. Status(M) =200 Status(M) =3xx Stop Go to Policy 2 Stop Yes No

22 Policy one: URI-R with HTTP redirection Evaluation: o Policy scope has: 1471 URIs (that have live redirection) o 77 out of 1471 have no mementos at all o 17 out of 77 have been retrieved mementos based on live redirection

23 Policy two: URI-M with HTTP redirection http://www.cnn.com/ Accept-Datetime: Sun, 13 May 2006 http://www.cnn.com/ http://www.cnn.com/

24 Policy two: URI-M with HTTP redirection Evaluation: o Policy scope: 2980 TimeMap (that showed HTTP redirection status code in at least one memento) o Success criteria: Using policy two contributed to the original TimeMap o Success percentage: 58% of the cases

25 Conclusion Quantitative study with 10,000 URIs. 48% were not fully stable through time. 27% were not perfectly reliable through time. New archival retrieval policy: o Policy one: successfully retreived mementos for17 out of 77 o Policy two: Expanded the timemap for 58% of cases. aalsum@cs.odu.edu @aalsum

Download ppt "Archival HTTP Redirection Retrieval Policies Temporal Web Analytics Workshop 2013, Rio De Janiro Ahmed AlSum, Michael L. Nelson Old Dominion University."

Similar presentations

A Web-Based Resource Model for eScience: Object Reuse & Exchange 2008 Microsoft eScience Conference Indianapolis, December 8, 2008.

A Web-Based Resource Model for eScience: Object Reuse & Exchange 2008 Microsoft eScience Conference Indianapolis, December 8, 2008.
Hiberlink is funded by the Andrew W. Mellon Foundation Investigating Reference Rot in Web-Based Scholarly Communication Martin Klein Los Alamos National.

Hiberlink is funded by the Andrew W. Mellon Foundation Investigating Reference Rot in Web-Based Scholarly Communication Martin Klein Los Alamos National.
Its all about sharing Sylvia van Peteghem, Ghent University.

Its all about sharing Sylvia van Peteghem, Ghent University.
Towards Building a Collection of Web Archiving Research Articles Brenda Reyes Ayala and Cornelia Caragea University of North Texas

Towards Building a Collection of Web Archiving Research Articles Brenda Reyes Ayala and Cornelia Caragea University of North Texas
Using the Memento MediaWiki Extension to Avoid Spoilers Shawn M. Jones Old Dominion University.

Using the Memento MediaWiki Extension to Avoid Spoilers Shawn M. Jones Old Dominion University.
Using OAI-PMH for Resource Exchange OAI Metadata Harvesting Workshop, JCDL 03 Michael L. Nelson, Terry L. Harrison Old Dominion University Norfolk VA

Using OAI-PMH for Resource Exchange OAI Metadata Harvesting Workshop, JCDL 03 Michael L. Nelson, Terry L. Harrison Old Dominion University Norfolk VA
Open Annotation Collaboration Rob Sanderson, Herbert Van de Sompel DMSS Meeting, May 14-15, Stanford, CA Robert Sanderson –

Open Annotation Collaboration Rob Sanderson, Herbert Van de Sompel DMSS Meeting, May 14-15, Stanford, CA Robert Sanderson –
Operational Auditing--Fall 200810-1 Evidence = Support n Auditors collect information n Evidence should be sufficient, competent, relevant & useful n Methodology.

Operational Auditing--Fall Evidence = Support n Auditors collect information n Evidence should be sufficient, competent, relevant & useful n Methodology.
© 2001 Franz J. Kurfess Introduction 1 CPE/CSC 580: Knowledge Management Dr. Franz J. Kurfess Computer Science Department Cal Poly.

© 2001 Franz J. Kurfess Introduction 1 CPE/CSC 580: Knowledge Management Dr. Franz J. Kurfess Computer Science Department Cal Poly.
THIS IS THE TITLE OF MY POSTER. THIS IS THE TITLE OF MY POSTER NAMES, NAMES, & NAMES This is my abstract. This is my abstract. This is my abstract. This.

THIS IS THE TITLE OF MY POSTER. THIS IS THE TITLE OF MY POSTER NAMES, NAMES, & NAMES This is my abstract. This is my abstract. This is my abstract. This.
Prototypes of pro-active approaches to support the archiving of web references for scholarly communications Richard Wincewicz 1, Peter Burnhill 1 & Herbert.

Prototypes of pro-active approaches to support the archiving of web references for scholarly communications Richard Wincewicz 1, Peter Burnhill 1 & Herbert.
EVALUATING THE TEMPORAL COHERENCE OF ARCHIVED PAGES SCOTT G. AINSWORTH MICHAEL L. NELSON OLD DOMINION UNIVERSITY IIPC 2015 APRIL 27 – MAY 1, 2015 STANFORD.

EVALUATING THE TEMPORAL COHERENCE OF ARCHIVED PAGES SCOTT G. AINSWORTH MICHAEL L. NELSON OLD DOMINION UNIVERSITY IIPC 2015 APRIL 27 – MAY 1, 2015 STANFORD.
The Water Cycle Task Process Evaluation Conclusion Introduction

The Water Cycle Task Process Evaluation Conclusion Introduction
July 25, 2012 Arlington, Virginia Digital Preservation 2012warcreate.com WARCreate Create Wayback-Consumable WARC Files from Any Webpage Mat Kelly, Michele.

July 25, 2012 Arlington, Virginia Digital Preservation 2012warcreate.com WARCreate Create Wayback-Consumable WARC Files from Any Webpage Mat Kelly, Michele.
Memento Update CNI Task Force Meeting, Spring 2011 1 Memento Herbert Van de Sompel Robert Sanderson Michael L. Nelson Giant Leaps.

Memento Update CNI Task Force Meeting, Spring Memento Herbert Van de Sompel Robert Sanderson Michael L. Nelson Giant Leaps.
Persistent Annotations Deserve New URIs Abdulla Alasaadi Old Dominion University Michael L. Nelson JCDL 2011 Ottawa,

Persistent Annotations Deserve New URIs Abdulla Alasaadi Old Dominion University Michael L. Nelson JCDL 2011 Ottawa,
OAI-PMH for Resource Harvesting Tutorial OAI4, October 20 th 2005, CERN, Geneva, Switzerland OAIResource Software Her This work supported in part by the.

OAI-PMH for Resource Harvesting Tutorial OAI4, October 20 th 2005, CERN, Geneva, Switzerland OAIResource Software Her This work supported in part by the.
Web HTTP Hypertext Transfer Protocol. Web Terminology ◘Message: The basic unit of HTTP communication, consisting of structured sequence of octets matching.

Web HTTP Hypertext Transfer Protocol. Web Terminology ◘Message: The basic unit of HTTP communication, consisting of structured sequence of octets matching.
Dec 9-11, 2003ICADL 20031 Challenges in Building Federation Services over Harvested Metadata Hesham Anan, Jianfeng Tang, Kurt Maly, Michael Nelson, Mohammad.

Dec 9-11, 2003ICADL Challenges in Building Federation Services over Harvested Metadata Hesham Anan, Jianfeng Tang, Kurt Maly, Michael Nelson, Mohammad.
TEMPORAL SPREAD IN ARCHIVED COMPOSITE RESOURCES (WORK IN PROGRESS) SCOTT G. AINSWORTH MICHAEL L. NELSON OLD DOMINION UNIVERSITY COMPUTER SCIENCE WADL 2013.

TEMPORAL SPREAD IN ARCHIVED COMPOSITE RESOURCES (WORK IN PROGRESS) SCOTT G. AINSWORTH MICHAEL L. NELSON OLD DOMINION UNIVERSITY COMPUTER SCIENCE WADL 2013.

Similar presentations

Ads by Google