1. Web Pages Seamless Flow Analysis Phase Programme Board December 2004

2. Seamless Flow Analysis Phase Web Pages Agenda 1.Introduction 2.State of play 3.The products Process model Data model Requirements catalogue 4.Summary and close

3. Seamless Flow Analysis Phase Web Pages Terms of Reference Objectives 1.Provide the business scoping for the Seamless Flow constituent projects, in terms of high-level requirements. 2.Provide a starting point for projects to perform detailed analysis and design, through initial process and data modelling. Deliverables High-level requirements catalogue. Detailed process model, using the high-level process model from the previous (initiation) phase as a starting point. Seamless data model, covering the scope as defined by the requirements and process model.

4. Seamless Flow Analysis Phase Web Pages InitiationKick-offFinaliseAnalysis Collate material Requirements Catalogue Process / Data Models Share with Board PRODUCTS Consolidate Review WS Review WS Review WS Review WS Review WS How the plan fits together… Modelling WS Modelling WS Modelling WS Modelling WS Modelling WS Reqts WS Reqts WS Reqts WS Themes Reqts WS Reqts WS

5. Seamless Flow Analysis Phase Web Pages (the life cycle of a record) 1. Record Usage at OGD 5. Technology Watch 4. Resource Discovery and Presentation 3. Preservation and Maintenance 2. Ingest record TNA Sources Payment system Record copying FOI Tracking system Users Customer registration PRONOM suite ExternalsTNA staff Paper processing TNA staff Inventory system Closure Control system Ordering system OGDs Paper system Process Model – Top level

6. Seamless Flow Analysis Phase Web Pages 1. Record Usage at OGD 1.1 Set disposal criteria (OGD) TNA 1.6 Check for and obtain FOI exemptions (OGD) 1.4 Selection 1.3 Appraisal 1.5 Sustain record (OGD) 2.2 Physical transfer 1.2 Create new record (OGD) 1.8 TNA technology watch advice 1.7 Apply FOI redactions where required (OGD) 1.9 Prepare for transfer (OGD) Wild West Starting point Record Metadata (Wild West) Record Metadata (Non Wild West) Record Metadata (Wild West) Custody policy Record freeze (Semi-Current – 10-30) Record Metadata (migrated / preserved) Record Metadata Check application (Close if applicable) Finished Records (releasable or Redacted) Biz classn / file plan Standards & guidance OSPs Custody policy Inventory File plan Filing guidance Non Wild West Starting point Appraisal criteria OSPs Technical report (DPD) Appraisal data Enhanced technical report Metadata Advice Record repository / inventory Record Metadata Accession decision Record Metadata (business usage of record prior to transfer to TNA)

7. Seamless Flow Analysis Phase Web Pages 2. Ingest Record 2.5 Record Salvage 2.3 Pre- Accession loading 2.2 Physical transfer 1.9 Prepare for transfer 3.1 Update archival descriptions 2.1 Prepare for transfer 2.4 Reconciliation OGD Liaison (Agree date, Mechanism, media) Metadata for preliminary look Notification, incl. Security level info Paper records (hybrid accession) Paper system Invalid record Notification of receipt (incl. incomplete, surprises, etc.) Approval to load Record, metadata Media Supporting material Accession id Consignment list Approval to load Record, metadata 5.1 Identify technical charactn Alert (problems) Archival description trigger Notification of validity Recommendation 3.2 Store record and metadata Record Metadata 1.9 Prepare for transfer Metadata (for closed records) (making sure we have a record to work with)

8. Seamless Flow Analysis Phase Web Pages 3. Preservation and Maintenance 3.2 Maintenance 3.1 Accessioning 2. Ingest Record 5. Technology Watch Record Metadata 4. Resource Discovery and Presentation Record Metadata Query, trigger Migrated records Record details Records Presentation copy (working with the record)

9. Seamless Flow Analysis Phase Web Pages 3.1 Accessioning 3.1.2 Store record and metadata 3.1.4 Transfer presentation copies to presentation areas 3.1.1 Update archival descriptions Stage 1 notification Ordering system 3.1.3 Identify / Create presentation copies Trigger back that lodging completed Mandatory metadata fields completed (Accession route) OGDs Base preservation copy + metadata to manage / describe objects Presentation copy (Accession route) Available Release trigger to catalogue for paper records 3.1.6 Completion of accessioning 2.3 Pre- Accession loading Archival description trigger Metadata 2.3 Pre- Accession loading 2.4 Reconciliation Record Metadata Approval to proceed Paper processing Paper record details (part of hybrid) Inventory system Presentation copies 4.4 Supply TNA record Presentation copy Stage 2 notification Trigger back that Paper lodging complete 3.2 Maintenance Record Metadata 3.1.5 Control closure Metadata, Record Updated metadata Closure control system Request Closure info Updates (prior to agreeing Transfer date)

10. Seamless Flow Analysis Phase Web Pages 3.2 Maintenance TNA staff 3.2.7 Generate new copies 3.2.3 Identify / Create presentation copies 3.2.1 Update archival descriptions 3.2.2 Store record and metadata OGDs Migrated record Base preservation copy 3.2.4 Transfer presentation copies to presentation areas 3.1 Accessioning Technology watch driven need for change (record request) 3.2.5 Control closure Metadata, Record Updated metadata Closure control system Request Closure info 3.2.8 Redaction updates Updates (prior to agreeing Transfer date) Presentation copies 4.4 Supply TNA record Presentation copy Redaction details 3.2.9 Controlled TNA access Viewable copy OGDs Request for closed records Closed records Metadata Update catalogue trigger Open record trigger 3.2.6 Extract copy Record 5.5 (Auto) migration Record Metadata 5.1 Identify Technical characterisation Record Metadata Records (to migrate) 5.5 (Auto) migration 5.2 Identify changes and impact on records stored Record query List of records affected (of type) New manifestations (migrated records)

11. Seamless Flow Analysis Phase Web Pages 4. Resource Discovery and Presentation 4.3 Payment 4.4 Supply TNA record 4.2 Request record 4.1 Discovery and search Agreed estimate Method of payment Distant Selling Regulations Contact details Sources Other records (Portal / link) Contact us Record copying Tools feedback Payment system 5.1 Identify technical characterisation 4.5 Add resource discovery metadata FOI Tracking System User 3.1.6 Completion of accessioning 3.2.4 Transfer presentation copies to Presentation areas Results Record Ref TNA staff Record Metadata Method of access Chosen search result Access request Receipt Record Trigger production of delivery copy Financial interface Hyperlinks (from other web sites) Updates Presentation copy 5.5 (Auto) migration New version available 5.4 Technology refresh planning Cannot do Request Users Customer registration User entry point Request Registration (record delivery to the users)

12. Seamless Flow Analysis Phase Web Pages 5. Technology Watch 5.1 Identify technical characterisation Externals 5.5 (Auto) migration 5.4 Update technical information 5.2 Identify changes and impact on records stored 5.3 Technology refresh planning 4.4 Supply TNA record 4.1 Discovery and search 3.2.2 Store record and metadata Technical info Exception report Technical info Problems (no viable plan) Alert Metadata Impact / Requirements Request Cannot do Migration pathway Selected records Metadata Load new migrated Manifestations back (migrated records) New version available Record Metadata 2.3 Pre- Accession loading Updates to technical repository Alert (problems) Enhanced metadata Exceptions for resolution PRONOM suite TNA staff Migration QA 3.2.2 Store record and metadata Record query List of records affected (of type) 3.2.2 Store record and metadata Record request (trigger) Records (to migrate) 3.2.2 Store record and metadata (Ensuring future record accessibility) 4.4 Supply TNA record Tools feedback Request Cannot do

13. Seamless Flow Analysis Phase Web Pages Data model Metadata OGD Business Unit (Series) Search Result Customer Order Holding Body Object Record Accession Plan Record Component Medium / Delivery Channel Deliverable / Presentation Unit Manifestation Action Audit Access Control Policy Selection Criteria Division Supporting Documentation / Resource Record Breakdown Reason Technical Info Pathway Validation Action Exemption Info Access Info Notification Application System TNA Staff Record Migration Action Technical Info Source Manifestation Type 2 4 5 3 1

14. Seamless Flow Analysis Phase Web Pages TNA Objectives 1 1.To develop effective and efficient management of active records to support government business operations 2.To improve the selection of public records for permanent preservation for the benefit of the public and of government 3.To ensure that our policies for records selection are socially inclusive 4.To make arrangements for the permanent preservation of government digital records of enduring historical value; and to ensure that archives on traditional media are kept in accordance with national and international standards 5.To extend the range and improve the quality of our online public services 6.To improve the quality and efficiency of our onsite public services Process requirements contribution to the objective 1, 4, 5 1, 4 1, 4, 5 3, 4, 5 1, 2, 3, 4, 5 3, 4, 5

15. Seamless Flow Analysis Phase Web Pages TNA Objectives 2 7.To promote the public records as a national information and learning resource to the widest possible audience 8.To raise an awareness of our holdings among under-represented groups in our user profile by means of a rigorous social inclusion programme 9.To promote awareness of, and access to, archival sources relating to British history 10.To develop standards and guidance for archive and records management professionals and to promote and monitor their use 11.To contribute to the implementation of the Government Archives Policy Action Plan and to monitor progress towards its objectives 12.To raise the public profile of The National Archives Process requirements contribution to the objective 1, 2, 3, 4, 5 3, 4 4 4 5 4, 5 ?!

16. Seamless Flow Analysis Phase Web Pages Requirements Summary 1 Record Usage at OGD Disposal / selection criteria Partial transfers Advice and guidance Ongoing validation of disposal criteria Taking in redactions Secure storage for FOI Facilitate OGD migrations

17. Seamless Flow Analysis Phase Web Pages Requirements Summary 2 Ingest Record Record reference Transfer mechanism / custody notifications Security classification aspects Check records and metadata (and quarantine) Air-gapped systems Progress / status reporting

18. Seamless Flow Analysis Phase Web Pages Requirements Summary 3 Preservation and Maintenance Archival description Closure checks Automated metadata loading Processing records and notifications Referential integrity of records and metadata Capacity planning QA and audit trail Management of multiple copies Back-up and recovery

19. Seamless Flow Analysis Phase Web Pages Requirements Summary 4 Resource Discovery and Presentation Resource discovery metadata Various access methods (e.g. browse and search) Metadata and full record text search Record ordering and copyright issues Chargeable vs. non-chargeable KIS Display, help, etc. features View on-line or at Kew only Controlled access to closed records

20. Seamless Flow Analysis Phase Web Pages Requirements Summary 5 Technology Watch Formats guidance to OGDs Automated format identification and validation Technical object inter-relationships Generate obsolescence alerts Migration plan with various stages, or pathways Migration plans should support automated migrations Reporting and control of failed migrations Process controls

21. Seamless Flow Analysis Phase Web Pages TNA records management monitoring Custom requests and delivery options Pricing and payments Login accounts Added value services One shopping basket Dummy presentation copy for testing Audit trails Multiple migration pathways External refereeing re quality of data (archivists / specialists) Multiple formats Manual checks Requirements Out of Scope

22. Seamless Flow Analysis Phase Web Pages More Requirements Out of Scope? Advice to OGDs on version control for redactions Encourage reduction in formats Accessibility and disability Backtrack manifestation migrations (dead-ends and failures) Choice of delivery (display / download) Tools for OGDs (format id) and externals (format info) Separate preservation / presentation migration plans

23. Seamless Flow Analysis Phase Web Pages Summary Process model –Understand detail (inputs, outputs, controls, resources), incl. threads –Descriptions make up the Seamless Flow process Data model –Record vs. metadata –Project specific vs. programme-wide –Attributes are a starter Requirements catalogue –Developed detail –Some outstanding questions –443 (8% OOS – over half in Resource Disc & Presentation)

24. Seamless Flow Analysis Phase Web Pages Process : Project basic mapping 1. Record usage at OGD 2. Ingest record 3. Preservation and maintenance 4. Resource discovery and presentation 5. Technology watch Departmental FOI Activities Appraisal and Selection Management of Semi-current Records Ingest Process (Transfer to TNA) Preservation Cataloguing (Metadata, archival description) Management and Security Cataloguing (Resource discovery) Presentation Portal Technology Watch Business Change Training

25. Seamless Flow Analysis Phase Web Pages Going forward Final feedback Project scopes / profiles Address outstanding issues Information Architect Programme (and project) planning

26. Web Pages Thank You