Svein Arne Brygfjeld National Library of Norway Nordic Web Archive

The message of today First: A summary Second: Legal deposit in Norway Third: Our digital library principles Fourth: Harvesting, archiving and giving access to the web Fifth: The prototype, a demonstration

Part one: Summary Norwegian legislation on legal deposit: Includes digital information! The national library of Norway has a relatively advanced digital library activity Nordic cooperation on methods and technology for legal deposit of the web Nordic project on access to web archives

Part Two: Legal deposit in Norway Legislation revised in 1989 Includes all information carriers in the ”traditional domain”, like books, newspapers & more Also including music and broadcast programs And: Including the information living in the digital domain

The National Library of Norway Bendik Rugaas Administration IT & Innovation National Librarian Rana Division Oslo Division 200 employees Administration IT Technical Repository Legal Deposit Media Lab Sound & Image 100 employees Administration IT Public Collections Bibliographic Norwegian Music (Svein Arne) 2

The challenge: Preserving the cultural heritage represented by the world-wide web –Including harvesting and archiving Giving access to historical web archives –…Nordic Web Archive access project

But first: Part three Our digital library principles…

One strategy for most digital objects One large long-term digital repository All storage, long-term preservation and access based on this infrastructure

Our Digital Library reference model -unix servers - fault tolerant disk systems -Tape libraries -HSM -Search Engines -Personalization -Specialized applications -Collecting applications -Metadata (DC) -Identification (URN) -Migration -Quality and Formats -IPR/Copyrights/Access control - text, audio, still images, moving images, web pages & more General storage facility Digital objects Repository functionality & organization Digital Library application layer

Examples of current use Digital Radio Archive –Digitization & archiving of hrs Galleri NOR –Still images in high quality Historical news-papers –Images of pages as well as OCR-based text

And now… …the preservation of the web!

Preserving the web: some focus areas Harvesting & collecting it all Archiving –Identification, versions, metadata, long- term preservation Access to archive

Harvesting Can it be possible? –Have a look at the search engines Available software –Public domain/OpenSource NEDLIB –Commercial several

Harvesting: Resolution in time Snapshots vs continous Continous: –Wanted for services considered interesting and with rapid updates –Dependent on use of software agents placed at the publisher

Everything or bits & pieces Questions to be answered: –What is (technically) possible? –What do we want? –What level of metadata do we need?

Archiving Different models in the five countries (probably) The norwegian model based on use on the library’s general storage facilites Close integration to other digital objects Online or near-line

Long-term preservation Migration –So far our choice Emulation –Technically complicated Museum –Hard to do over time

And now… …access to web archives

Nordic Web Archive A context for cooperation to find common technology and methods to harvest, archive and give access to the web Current focus on access to archives –Small, focused project

NWA: Members Denmark (Royal Library) Finland (National Library) Iceland (National Library) Norway (National Library), project mgmt Sweden (Royal Library) Nordunet2

NWA: Current scope Focus on access to web archives NOT harvesting NOT archiving

NWA: Main choises General and well-specified interface to archive Search (and navigation) through the use of a commercial search engine Access based on search and navigation/browsing Support for navigation in time and space

NWA: Architecture XML COMMON FORMAT INDEXES WEB INTERFACE ARCHIVE ACCESS SEARCH ENGINE INDEXER FIND_DOCUMENT(URN) DOCUMENT FIND_ID (URL,TIME) URN

NWA: The technology Based on commercial search engine from Fast Search & Transfer In-house development on Linux-platform –XML, PHP, Perl and Java –Probably OpenSource –General web user interface (no additional plugins needed)

NWA: Search engine motivations Motivation –Support for search functionality on text documents –Speed –Reduced complexity in implementation

NWA: Search engine benefits (in addition to fullfilling the motivations) –Extreme scalability –Support for distributed searching –Easy integration with other indexes –Integrated language technologies (limited)

NWA: Access methods Main principles: –The web seen in the archive should look like it did on the net –It should be available through the use of a ordinary web browser Three main methods –Search, navigation and browsing

NWA: Search Search based on search engine Indexes based on exports from archives –In general search on the original content is possible, but –Some additional information available Protocol metadata, timestamps and more Time limitations, phrase search and other funtionalities

NWA: Search cont.

NWA: Time navigation Given a location or service –The user should easily be able to go to next/previous version Using a JAVA-based time-line as time navigation tool

NWA: Time navigation cont.

NWA: Space navigation Given a point of time –The user should be able to go some other service based on the url In NWA prototype, the user can use original url’s as reference to service within the archive

NWA: Space navigation

NWA: Metadata Few web recources contain user-produced metadata HTTP contains some metadata, like time of modification and more Tagging of documents (like ) can be viewed as metadata, and is passed on to the indexer

NWA: Open Source? Many good reasons pro, few contra Dependent on third-party software! –Radical re-implementation to be independent

NWA: Scalability Search engine extremely scalable

Further challenges ”The deep web” Dynamic and user dependent services Continuity Description/metadata Access rights to archive! –This is the main obstacle

See also…

That’s it! Thank you for listening (if you were ;-) ) Please contact me if there’s anything –But on only!