1. Max Kaiser: PLANETS Testbed
The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference Lisbon, 5–6 September, 2007
Max Kaiser, Austrian National Library
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

2. Max Kaiser: PLANETS Testbed
ToC
Why do we need Testbeds in Digital Preservation?
What will the PLANETS Testbed deliver?
Role of Testbed within PLANETS
PLANETS Testbed services for Digital Preservation community
Testbed Methodology, Experiment 6-step Process
Testbed Software Application
Testbed Components
User Interface Mock-Up Demonstration
Next Steps
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

3. Why do we need Testbeds in Digital Preservation?
Max Kaiser: PLANETS Testbed
Why do we need Testbeds in Digital Preservation?
Concern about digital preservation rapidly increased in recent years
Several approaches to digital preservation implemented by various programs and projects
A number of digital preservation tools and services (e.g. for migration and characterization) developed
 Need for systematization, benchmarking and evaluation urgent in many fields:
Systematic analysis of current and potential preservation strategies
Evaluation and benchmarking of characterisation and migration tools
Assess the role of emulation tools …
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

4. Why do we need Testbeds in Digital Preservation?
Max Kaiser: PLANETS Testbed
Why do we need Testbeds in Digital Preservation?
If we want to
perform scientific research in digital preservation
evaluate preservation approaches in diverse “real life” settings
avoid duplication of work
maximise use of invested resources
we need a dedicated research environment,
that allows the systematic execution of experiments by different institutions
with experiments that
follow a formalized methodology
are reproducible
are fully documented and accessible for further analysis and comparison
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

5. A Testbed for Digital Preservation
Max Kaiser: PLANETS Testbed
A Testbed for Digital Preservation
Definition:
“A controlled environment for experimentation and evaluation, with metrics and benchmark content that allow comparison of preservation tools and strategies”
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

6. Max Kaiser: PLANETS Testbed
Platform for objective evaluation of preservation tools and strategies:
Load content
Perform digital preservation experiments:
Design the experiment, select data and tools, specify resources and evaluation criteria, run the experiment, evaluate results against criteria, compare outcomes
Find support for validation of preservation strategies
Benchmark tools and services
Will consist of
Hardware, data storage, PLANETS software, Testbed application
Benchmark and other content
Will provide services for
The PLANETS project partners
The preservation community
Tool developers and vendors
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

7. Role of Testbed in PLANETS | 1
Max Kaiser: PLANETS Testbed
Role of Testbed in PLANETS | 1
Test and validate the technical solutions and approaches developed in PLANETS:
provide a controlled hardware and software environment for testing and evaluating preservation action (migration, emulation) and characterisation tools and services
record experiments data in registries for further analysis and comparison
assist the validation of the effectiveness of different digital preservation plans
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

8. Role of Testbed in PLANETS | 2
Max Kaiser: PLANETS Testbed
Role of Testbed in PLANETS | 2
Assess the suitability of the approaches across „real life“ scenarios in various organisations
Analyse applicability of the outcomes of PLANETS in existing workflows and organisational contexts
Evaluate their efficiency in providing practicable solutions for organisations engaged in digital preservation
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

9. Role of PLANETS Testbed in the Digital Preservation Community
Max Kaiser: PLANETS Testbed
Role of PLANETS Testbed in the Digital Preservation Community
In a second phase, the PLANETS Testbed will
Offer services to organisations outside PLANETS:
Support institutions to test preservation tools and services against benchmark content
Assist institutions to validate their preservation plans against their policies & content profiles
Enable developers and third party vendors to submit tools for benchmarking and certification:
Validate the suitability of their tools in preservation workflows
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

10. PLANETS Testbed: Foundations
Max Kaiser: PLANETS Testbed
PLANETS Testbed: Foundations
Build on work undertaken by the Dutch Digital Preservation Testbed and DELOS Testbed Framework
Integration in PLANETS Interoperability Framework
Focus on formalisation of testing process design
Strong focus on comparability and traceability of results OK
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

11. Max Kaiser: PLANETS Testbed
Main Participants
Austrian National Library
Humanities Advanced Technology and Information Institute at the University of Glasgow (HATII)
Austrian Research Centers (ARC)
National Archives of the Netherlands
British Library
Vienna University of Technology
University of Cologne
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

12. PLANETS Testbed System
Max Kaiser: PLANETS Testbed
PLANETS Testbed System
Web based application
Provides a consistent and coherent evidence base for the objective evaluation of different tools, services, and complete preservation plans
Testbed system provides:
Clearly structured and formal process for preparing, executing, and evaluating experiments
Repeatability of experiments, comparability and traceability of results
Assistance in planning preservation strategies
Benchmarking of services
Results of ongoing experiments will improve strength of preservation plans by means of evaluation
Will provide essential feedback for the development of PLANETS tools, services and preservation planning
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

13. Max Kaiser: PLANETS Testbed
Testbed Methodology
Experiment Process:
Refined to a focused 6-step process that each formal experiment must follow
PLANETS Testbed experiment: consist of a consistent methodology for
Defining its metadata (exploration area, scope, criteria, references etc.)
Selecting or creating a workflow of (orchestrated) preservation services
Evaluating the results against the experiment’s initial goals
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

14. Max Kaiser: PLANETS Testbed
Start
1. Define Basic Experiment Properties
Basic Experiment Properties Definition
Testbed Experiment Process | 1
Name
Reference to other experiments
Summary
Purpose
Participants
Scope
Type of Tools
Type of Objects
Approach …
2. Design Experiment
Tools and Data Specification
Specify Experiment Type
Migration
Characterisation
Workflow
Specify Tools
Specify Data
3. Specify Resources & Outcomes
Evaluation Checklist
Resource Specification
Estimate number of output files
Estimate computer resource requirements
Specify benchmark goals for experiment
4. Go / No go
Automatic approval by system
Based on resource specification and system monitoring
Manual approval by administrator
Declaration of Go / No go
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

15. Max Kaiser: PLANETS Testbed
Testbed Experiment Process | 2
Experiment Workflow
Experiment Results
4. Go / No go
5. Run Experiment
Experiments executed as BPEL workflows
Processed by workflow execution engine of the Interoperability Framework
Presentation of status report
6. Evaluate Experiment
Experiment Evaluation
Presentation of input and output data
Evaluation of benchmark goals
Stop
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

16. Max Kaiser: PLANETS Testbed
Testbed System Design
Initial understanding of the required characteristics of PLANETS Testbed derived from the Dutch Preservation Testbed and the DELOS Digital Preservation Cluster
Additional requirements gathered following a clearly structured software design process
Interviews
User Scenarios  ~ 60 Use Cases
Software Requirements
IF Requirements (Authentication Interfaces, Logging, Versioning…)
Perform Experiment, Define Basic Properties, Select A Tool, View Available Services, Handling Stalled Experiments…
Non-functional requirements
Usability, Reliability, Performance, Security, Supportability…
Design and Implementation Plan
High-level design of the Testbed application
Initial breakdown of the system into components
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

17. Max Kaiser: PLANETS Testbed
Testbed Software
Platform-independent system based on Java Enterprise Edition (EE)
Three-tiered system:
Handles presentation of user interfaces; logic for dynamically creating content. Based on Java Server Faces and Facelets
Provides business logic (core functionality). Based on JavaBeans and JAX web service endpoints
Provides data persistency. Uses data registries provided by PLANETS Interoperability Framework
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

18. Max Kaiser: PLANETS Testbed
IF Applications
Workflow
Designer
Administration Tool
Testbed Application
Experiment
Designer
Evaluator
Im/Exporter
Data
Manager
Services
Service
General
Administration
Browser
Integrate
System Monitoring Client
Persistence
Scheduling
Approval
TB Service Registry
Client
TB Data Registry
Testbed
Backend
Components
Use
Configure
Interoperability
Framework
Service Bus
Security/Authentication/ Authorization
Monitoring/Logging/Auditing
Workflow Execution Engine
Error/Exception Handling
Transaction Manager
Database Layer
Work
Space
Service
Registry
Services
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

19. Experiment Designer component | 1
Max Kaiser: PLANETS Testbed
Experiment Designer component | 1
Testbed Application
General
Experiment
Data
Services
Designer
Evaluator
Im/Exporter
Browser
Manager
Data
Manager
Service
Administration
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

20. Experiment Designer component | 2
Max Kaiser: PLANETS Testbed
Experiment Designer component | 2
Guides the user trough design of an experiment
User Interface for steps 1–3 of the experiment process
Define basic properties
Design experiment
Specify resources
Select whether experiment is formal or informal
Select type of experiment
Migration
Characterisation
Emulation
Preservation plan evaluation
Execution of a predefined workflow
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

21. Experiment Designer component | 3
Max Kaiser: PLANETS Testbed
Experiment Designer component | 3
Interaction with several other Testbed components, most notably the service manager and data manager
Interaction with Service Manager
Browse PLANETS Service Registry
Selection and configuration of (wrapped) tools for Preservation Action (PA) and/or Preservation Characterisation (PC)
„Service“: PA/PC tool exhibiting well-defined functionality through a PLANETS IF compliant web service wrapper
Interaction with Data Manager
Interface for handling data input and output via API for accessing the IF Data Registry
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

22. Experiment Designer component | 4
Max Kaiser: PLANETS Testbed
Experiment Designer component | 4
Specification of Resources
E.g. number of output files
Specification of Outcomes of an experiment
Will appear at the evaluation stage of experiment
Experiment stored as BPEL file and handed over to Approval component
Additional features:
Loading existing experiments
Viewing of experiments in “passive” mode (read-only)
Commenting on experiments
Re-use and Repeating experiments
Re-use certain aspects of existing experiments, or
Repeat entire experiments with original data
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

23. Experiment Designer component | 5
Max Kaiser: PLANETS Testbed
Experiment Designer component | 5
Rendering of BPEL workflows
All experiments are executed as BPEL workflows
“Simple” migration and characterisation experiments:
BPEL files are created within Testbed based on templates
Up to three orchestrated services (e.g. characterisation followed by migration followed by characterisation)
Complex workflows are created by using the IF Workflow Designer (or any other BPEL tool) and imported into the Testbed
Component does not process the actual execution of an experiment!
Handled within the PLANETS Interoperability Framework by the Workflow Execution Engine
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

24. Experiment Approval & Scheduling | 1
Max Kaiser: PLANETS Testbed
Experiment Approval & Scheduling | 1
Testbed Application
General
Experiment
Data
Services
Administration
Designer
Evaluator
Im/Exporter
Browser
Manager
Data
Manager
Service
Scheduling
Approval
System Monitoring Client
Testbed
Backend
Components
Persistence
TB Service Registry
Client
TB Data Registry
Client
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

25. Experiment Approval & Scheduling | 2
Max Kaiser: PLANETS Testbed
Experiment Approval & Scheduling | 2
Approval component handles technical approval of experiments by the system and/or the administrator
Makes use of information provided by the experimenter and by the system monitoring component – e.g.:
Number of input files
Estimated number of output files
Current CPU / Memory usage
Threads / Processes
Available Disk Space
Scheduling component allows administrator to
Select a date/time for an experiment to start
Assign a maximum run time
Check progress of experiments
Stop experiments
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

26. Experiment Evaluation | 1
Max Kaiser: PLANETS Testbed
Experiment Evaluation | 1
Testbed Application
General
Experiment
Data
Services
Designer
Evaluator
Im/Exporter
Browser
Manager
Data
Manager
Service
Administration
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

27. Experiment Evaluation | 2
Max Kaiser: PLANETS Testbed
Experiment Evaluation | 2
View files and results that have been produced during the experiment
Capture experiment conclusions in a structured way
Evaluate results based on the criteria defined during experiment design process
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

28. Experiment Importer / Exporter | 1
Max Kaiser: PLANETS Testbed
Experiment Importer / Exporter | 1
Testbed Application
General
Experiment
Data
Services
Designer
Evaluator
Im/Exporter
Browser
Manager
Data
Manager
Service
Administration
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

29. Experiment Importer / Exporter | 2
Max Kaiser: PLANETS Testbed
Experiment Importer / Exporter | 2
Create BPEL files from existing experiments and make them available for download
Upload BPEL files to the Testbed
Import whole experiments
Import predefined workflows as part of the experiments design process
Check validity of the BPEL file structure
Check data contained within BPEL files
Download experiment results
Information on experiments in PDF format created on-the-fly
Basic properties, tools used, data used, information about approval process, information about execution of experiment, experiments results, validation of experiment
 local demo
 WWW demo
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

30. Max Kaiser: PLANETS Testbed
Experimenter Pages
Login / Logout
Homepage
Browse Experiments
Browse Services
My Experiments
New Experiments
Browse Data
Import Experiments
View Experiment Details
Experiment Designer
View Metadata
Annotate Experiment
Export Experiment Setting
Upload Data
Run Experiment
Evaluate Experiment Results
Edit Conclusions
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

31. Conclusions / Next Steps
Max Kaiser: PLANETS Testbed
Conclusions / Next Steps
PLANETS Testbed: controlled environment for experimentation that allows benchmarking of preservation tools, services and strategies
First release to be expected in February / March 2008
Will be used to experiment with the preservation tools and services developed in PLANETS
In a 2nd Phase (by end 2008) Testbed will offer services to organisations outside PLANETS:
Support institutions to test preservation tools and services against benchmark content
Assist institutions to validate their preservation plans against their policies & content profiles
Main Testbed Instance hosted by University of Glasgow (HATII)
Dedicated hardware with sufficient processing power and storage space
Max Kaiser (Austrian National Library, Vienna): The PLANETS Testbed
DPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006

32. Max Kaiser: PLANETS Testbed
The PLANETS Testbed
Questions?
Max Kaiser, Austrian National Library
Thank you!
IST FP6 Digital Preservation Projects Meeting, Glasgow, 23 Nov 2006