1. A centre of expertise in digital information managementwww.ukoln.ac.uk Facing The Challenges Of A Standards-Based Approach To Web Development Brian Kelly UKOLN University of Bath Bath Email B.Kelly@ukoln.ac.uk UKOLN is supported by: http://www.ukoln.ac.uk/web-focus/events/conferences/ili-2005/talk-2/ About This Talk This talk will not tell you that open standards are great and you should just use them, but address the difficulties we face in making use of standards About This Talk This talk will not tell you that open standards are great and you should just use them, but address the difficulties we face in making use of standards

2. A centre of expertise in digital information managementwww.ukoln.ac.uk 2 Contents This talk will cover: Why we need open standards The problems with open standards Ideology or pragmatism? A layered approach:  A contextual approach for standards and related processes – for funding bodies  A contextual approach for development – for funding bodies Extending this approach Conclusions

3. A centre of expertise in digital information managementwww.ukoln.ac.uk 3 Background Many digital library programmes require use of open standards But it appears that there is little policing of compliance with open standards. Some argue for more rigourous policing But issues are not always clear-cut: Uncertainty of the meaning of open standards Immaturity of standards Lack of support from tools Flexibility of marketplace solutions Costs This talk seeks to address these challenges and provide an achievable approach Why Standards?

4. A centre of expertise in digital information managementwww.ukoln.ac.uk 4 Examples JISC & UK HE Digital Library Programmes Standards document produced for eLib programme (1994) and updated to support recent digital library programmes The NOF-digitise Programme Standards document written to support the NOF- digitise programme for providing access to UK cultural heritage resources UK e-Government Standards framework developed to support e- Government work within central and local government Why Standards?

5. A centre of expertise in digital information managementwww.ukoln.ac.uk 5 Why Use Standards? In many digital library programmes there has been a philosophy based on use of open standards to: Avoid application lock-in and platform dependencies Minimise migration costs Provide long-term access to scholarly resources But in eLib programme (~1994-2000): Little policing of compliance with open standards Adoption of "let a thousand flowers bloom" This approach: Probably sensible approach in mid-1990s (Gopher?) Not desirable now:  Web is the killer application; XML is killer format  Need to maximise access; support M2M apps;...  Need to protect investment from public funding Why Standards?

6. A centre of expertise in digital information managementwww.ukoln.ac.uk 6 What Are Open Standards? But what are Standards and Open Standards? Are Flash & PDF formats and Java language open? Can we agree on the following characteristics: Standard ratified by recognised neutral standards body. An open standards-making process. Documentation is freely available on the Web. Use of the standard is uninhibited by licensing or patenting issues. Note that not all open standards bodies will comply with all of these features. The standards-making process within the W3C, for example, is initially restricted to organisations which are members of the W3C and a small number of invited experts. Why Standards?

7. A centre of expertise in digital information managementwww.ukoln.ac.uk 7 Surveying Our Communities Various surveys of Web sites have been carried out in order to monitor compliance with Web standards: Survey Of W3C Member Organisations Majority of W3C member home pages do not pass the W3C's compliance test (Feb 2003) See Survey Of Digital Library Programme Web Sites A survey of 50+ home pages for JISC's 5/99 programme was carried out in 2003 Poor compliance found Other Surveys Other surveys of UK museums, libraries, FE colleges give similar findings Problems With Standards

8. A centre of expertise in digital information managementwww.ukoln.ac.uk 8 Difficulties With Open Standards Why do bodies which seek to use open standards experience such difficulties? Lack of awareness of importance of standards Difficulties in implementing standards Difficulties in checking compliance Immaturity of the standards Limitations of the standards Lack of support from tools … Problems With Standards

9. A centre of expertise in digital information managementwww.ukoln.ac.uk 9 Can We Trust The Standards Guys? RSS A powerful light-weight syndication standard Rich Site Summary, RDF Site Summary or Real Simple Syndication? RSS 1.0 spec maintained by self-confessed ‘teenager writer, hacker and activist”! RSS 2.0 developed by interesting personality and now hosted at Harvard Law School Where’s the neutral trusted standards agency? Web Services / Semantic Web / … “Web Services Considered Harmful?” Complexity of Semantic Web Are new standards complex & over-engineered? Problems With Standards

10. A centre of expertise in digital information managementwww.ukoln.ac.uk 10 What Do We Do? What approaches should we be taking? Surrender To The Proprietary World Should we allow public sector resources to be developed in proprietary formats? Stronger Promotion/Enforcement Of Standards? Groups such as W3C's QA activity and the Web Standards projects, … feel we should be promoting standards-compliance work more forcefully But will this single-minded approach work; it appears to ignore complexities of the real world Solutions

11. A centre of expertise in digital information managementwww.ukoln.ac.uk 11 An Open Standards Culture And A Layered Approach There is a third way! The development of an open standards culture: Promotes the benefits of open standards Promotes exemplars showing best practices in use of open standards whilst: Recognising difficulties of compliance Recognising challenges of resourcing, technical expertise, … A layered model based on these principles has been developed to support JISC’s digital library programmes Solutions

12. A centre of expertise in digital information managementwww.ukoln.ac.uk 12 A Layered Approach (1) A layered approach has been developed Solutions

13. A centre of expertise in digital information managementwww.ukoln.ac.uk 13 A Layered Approach (2) Context: Contextual layer allows for variability: (Middleware must comply with xyz; reports can use MS Word; …) Policies: Catalogue of standards, with risk assessments Other areas covered e.g. software; accessibility; … Compliance: Compliance regime, such as: External checking / penalty clauses Self-assessment  Reporting of exceptions Quality Assurance: QA to ensure solutions are implemented correctly: documented policies/systematic checking procedures Solutions

14. A centre of expertise in digital information managementwww.ukoln.ac.uk 14 Project’s Perspective The three-layered approach provides a model for use by the funding body, which helps the development of the standards catalogue The end user (the project) may not be aware of this model What will the project see? Solutions

15. A centre of expertise in digital information managementwww.ukoln.ac.uk 15 The Project’s Perspective Selection Ratification Quality Assurance Recommendations Environment Resources Formal Discussions Notification PoliciesCompliance ProceduresReporting Learning Case studiesRefining processBenefits to 3 rd parties Solutions

16. A centre of expertise in digital information managementwww.ukoln.ac.uk 16 Selection Process (1) As part of the selection process for standards there are several factors: Ownership and openness of standard (open, neutral body; proprietary but community process; community but spec publish; proprietary and reverse engineered; proprietary and closed)) Availability of viewers (multiple platforms; available for free; available as open source) Availability of authoring tools (multiple platforms; available for free; available as open source) Selection

17. A centre of expertise in digital information managementwww.ukoln.ac.uk 17 A Spectrum For Standards If we have defined open standards do we treat everything else (proprietary formats) equally bad? (And how should we regard PowerPoint users!) A Spectrum For Ownership: Is there a community process for standard development? Has the standard has been published openly Has the standard been reverse-engineered Java:Owned by Sun (open standardisation attempts aborted by Sun). However Community Process for development to language. PDF:Owned by Adobe. However specification has been published. Word:Owned by Microsoft. Specification has been reverse engineered. Note use of XML in next version – so is it an open standard? Java:Owned by Sun (open standardisation attempts aborted by Sun). However Community Process for development to language. PDF:Owned by Adobe. However specification has been published. Word:Owned by Microsoft. Specification has been reverse engineered. Note use of XML in next version – so is it an open standard? Selection

18. A centre of expertise in digital information managementwww.ukoln.ac.uk 18 Selection Process (2) Architectural Integrity (developed as part of broader framework – cf W3C specs) Fitness For Purpose (is the standard designed for the purpose envisaged) Expertise (does the organisation have the necessary expertise available in-house) Maturity of Standard (is the standard mature and well-proven) Local Culture (does the organisation seek to make use of emerging standards or prefer to use proven technologies) User Needs (does the standard satisfy the requirements of the user) Preservation Needs (is the standard appropriate for long-term preservation) Selection

19. A centre of expertise in digital information managementwww.ukoln.ac.uk 19 Selection Process (3) To assist the selection process we have developed an online and paper-based toolkit Note: The toolkit is intended to prompt discussion and not to provide answers Selection

20. A centre of expertise in digital information managementwww.ukoln.ac.uk 20 Ratification Once a project has been its selection of the standards to be used: The recommendation may need to be approved (funders, advisory boards, …) The funders may be informed of the recommendation The decision may be made by the project Note that this process can be applied in other areas (e.g. selection of software) Ratification

21. A centre of expertise in digital information managementwww.ukoln.ac.uk 21 QA – Policies & Procedures How do you know what you should do if you don't have documented polices? Policy: Web Standards Standard: XHTML 1.0 and CSS 2.0 Architecture: Use of SSIs and text editor Exceptions: Automatically-derived files Ownership: The project manager is responsible for this policy Checking: Use,validate after update Audit Trail: Use,rvalidate monthly and document findings Policy example You may find it useful to develop similar policies yourself – for example, a policy of the accessibility of your Web site Solutions

22. A centre of expertise in digital information managementwww.ukoln.ac.uk 22 Learning And Sharing Towards the end of a project, it can be helpful to have feedback on the process and the details: Was the standards catalogue useful? Did the process work? What changes are needed? As well as projects giving feedback, the decisions made by projects and their experiences can usefully by shared across the community: How did project X make use of RSS? What standards are project having problems with? … Such answers may be of use to other communities e.g. digital library programmes in other countries Solutions

23. A centre of expertise in digital information managementwww.ukoln.ac.uk 23 Embedding Within Programmes There is a need to embed this approach fully within digital library development programmes: Programme Infrastructure Not only covering standards, but also other aspects such as use of open source software, accessibility, …) Programme Calls Design of programme calls, evaluation process, … Project Development Work Selection processes; QA processes; … Project and Programme Termination Feedback mechanisms; maintenance of processes; … Further Developments

24. A centre of expertise in digital information managementwww.ukoln.ac.uk 24 Bidding Process Projects bidding for funding should demonstrate their acceptance of the open standards culture by describing : The standards they expect to use in their work The technical architecture which will be used The technical expertise they have to support this The QA procedures they will use in order to assess their compliance Their justification for use of proprietary solutions Further Developments

25. A centre of expertise in digital information managementwww.ukoln.ac.uk 25 Reporting Procedures Funded projects should demonstrate their use of open standards by providing the following information in their periodic reports: The standards they have implemented Use of proprietary formats The QA procedures they have implemented Audit trails showing compliance with standards Explanations of changes to original proposals Further Developments

26. A centre of expertise in digital information managementwww.ukoln.ac.uk 26 Applicability Elsewhere: Accessibility Layered approach can also be applied to guidelines WAI WCAG (Web Accessibility Content Guidelines) Are WCAG guidelines (which inform us on best practices) or standards (we must implement?) Not always appropriate to e-learning (may wish to make the answer difficult to find?), culture, art, … Notion of universal design runs counter to user- focused / inclusive design Poor WCAG compliance (in cultural heritage & education sector, disability bodies, etc.) can reflect WCAG limitations! Therefore: Use a contextual approach to accessibility and usability, … See W4A/WWW 2005 paper Further Developments

27. A centre of expertise in digital information managementwww.ukoln.ac.uk 27 Applicability Elsewhere: Software Use of open source software very relevant to digital library programmes But: Not all open source software is good May have reasons for using proprietary software So need for: Process for selection of software Mechanisms for choosing which OSS licence to use (if chosen) Approaches to software development … Work with OSS Watch in a consistent approach – see "Top Tips For Selecting Open Source Software" Further Developments

28. A centre of expertise in digital information managementwww.ukoln.ac.uk 28 Context Policies Compliance Software Context Policies Compliance Accessibility The Bigger Picture The vision is for: Modularity in approaches to standards, accessibility, software, digitisation,.. Context Policies Compliance Standards Evaluation Feedback Enhance Use Feedback mechanisms to help refine model and advice Support infrastructure for funders, projects & 3rd parties on model and best practices through sharing via briefing documents, case studies, events, maximising benefits through Creative Commons licences Projects3 rd Parties DocsEvents Best practices CC licences Funders Processes To On By Support

29. A centre of expertise in digital information managementwww.ukoln.ac.uk 29 Conclusions To conclude: Open standards are important in many digital library development programmes However use of open standards is not necessarily easy and may be costly Rather than abandoning open standards there is a need to adopt an open standards culture A layered approach can provide a mechanism for implementation This approach can be extended to other areas

30. A centre of expertise in digital information managementwww.ukoln.ac.uk 30 Questions Any questions?