1. QA Focus – Supporting JISC's 5/99 Programme The QA Focus Perspective Marieke Guy UKOLN M.Guy@ukoln.ac.uk http://www.ukoln.ac.uk/qa-focus/

2. QA Focus – Supporting JISC's 5/99 Programme The QA Perspective If you want to sort out the problems on your Web site you can do so by developing a Quality Assurance Framework This involves: Testing Problem tracking and corrective action Policies Procedures Documentation

3. QA Focus – Supporting JISC's 5/99 Programme What Can You Test? Functional testing Compatibility testing Load/performance testing Stress testing Usability testing Security testing Integration of unit testing Link testing HTML and CSS validation Reliability testing Regression testing …

4. QA Focus – Supporting JISC's 5/99 Programme Automated Testing Should be scalable Only suitable approach for large sites Can push (email) information about problems Many free testing tools available but: Will not spot all usability/accessibility problems Can provide too much information Comprehensive and configurable testing tools can be expensive

5. QA Focus – Supporting JISC's 5/99 Programme Manual Testing Can spot usability/accessibility problems Can make use of one’s community/user base/peers Can use of key pages which will identify problems which occur elsewhere Useful for testing new Web sites “5 users can spot most of your problems” but: Not suitable for testing large numbers of pages Quality of feedback may be variable Often not useful for testing established Web sites

6. QA Focus – Supporting JISC's 5/99 Programme Approaches to Automated Testing Automated testing can make use of: Web-based tools Desktop tools UKOLN has made use of Web-based testing tools: Monitor accessibility, HTML, CSS, compliance, page size, links quality, nos. of links, etc. Benchmarking approach provides comparison with one’s peers http://www.ukoln.ac.uk/qa-focus/surveys/

7. QA Focus – Supporting JISC's 5/99 Programme Peer Review “Give me comments on my Web site and I’ll also test yours” Can help in community-building Particularly useful for programmes in which project Web sites being developed at same time But: Can be difficult in processing unstructured responses Need to manage process

8. QA Focus – Supporting JISC's 5/99 Programme Problem Tracking There are three stages in a corrective action system: The short term fix The cure Prevention of errors – identify potential causes of error by analysing data to setect trends which if allowed to continue would result in errors

9. QA Focus – Supporting JISC's 5/99 Programme Documentation 3 Levels of documentation: Policy e.g. Quality Manual System e.g. Processes and procedures Work practices e.g. instructions More detailed, more specific, applies to fewer people

10. QA Focus – Supporting JISC's 5/99 Programme Policy A plan or course of action intended to influence and determine decisions Procedure A way of performing or effecting something Process A series of actions, changes, or functions bringing about a result Definitions

11. QA Focus – Supporting JISC's 5/99 Programme Documentation Good documentation Policies, procedures and audits Define procedures for all processes Requirements Specifications Mission statements or statement of direction Define your audience Document the testing suite and tools Document authoring tools etc. Document problems tracking A Quality Management System (QMS)??

12. QA Focus – Supporting JISC's 5/99 Programme Issues I Goals for testing will be related to the kind of site you have Testing time is limited – use automated tools Documentation is critical – to make sure practices are repeatable Implement your QA procedures slowly over time to avoid stifling productivity QA planning at the start of development means less time fixing things at the end

13. QA Focus – Supporting JISC's 5/99 Programme Issues II Motivating/persuading people to implement a QA Culture can be difficult – run workshops to show people what you are aiming for Make sure you have a feedback mechanism for the procedures you are implementing Having a great team/management is crucial

14. QA Focus – Supporting JISC's 5/99 Programme The QA Focus Perspective QA Focus: Funded by JISC Currently provided by UKOLN and AHDS Currently supports JISC's 5/99 programme Staff: Brian Kelly, Project manager Marieke Guy, QA Focus officer, UKOLN Hamish James, QA Focus officer, AHDS Gareth Knight, QA Focus officer, AHDS

15. QA Focus – Supporting JISC's 5/99 Programme Approach Taken Two possible approaches to ensuring compliance with standards and best practices: Enforce Inspect all project’s work Strict auditing, with penalties for no- compliance Encourage Training for project staff Developmental, explaining reasons for compliance, documenting examples of best practices and providing advice on implementation and monitoring The latter approach is preferable, especially in a HE context The latter approach is preferable, especially in a HE context

16. QA Focus – Supporting JISC's 5/99 Programme QA Focus Work Documentation and Advice Briefing Papers Case Studies FAQs Descriptions of tools and architectures for projects to implement best practices In-house QA Surveys Providing Motivation Workshops Liasing The QA Focus Toolkit

17. QA Focus – Supporting JISC's 5/99 Programme Documentation: Briefing Papers Advisory briefing documents are being produced These are: Brief, focussed documents Informed by findings of the surveys Advisory briefing documents are being produced These are: Brief, focussed documents Informed by findings of the surveys

18. QA Focus – Supporting JISC's 5/99 Programme Documentation: Case Studies Case Studies are being commissioned These are: Written by projects themselves Describe the solution adopted to a particular problem Include details of lessons learnt – not just a press release! Case Studies are being commissioned These are: Written by projects themselves Describe the solution adopted to a particular problem Include details of lessons learnt – not just a press release!

19. QA Focus – Supporting JISC's 5/99 Programme Web Site Testing Tools http://www.ukoln.ac.uk/qa-focus/qa-projects/ Full list of Web site testing tools available from database Surveys run with a number of the tools In the future there will be reviews of tools Full list of Web site testing tools available from database Surveys run with a number of the tools In the future there will be reviews of tools

20. QA Focus – Supporting JISC's 5/99 Programme Describing Tools and Approaches As an example of approach which provides easier access to testing tools, see the,tools approach deployed on UKOLN Web site. This approach: Applies to all resources on Web site Covers HTML and CSS validation and various other tests Some recursive apps (,rvalidate ) Easily implemented with single line redirect This approach: Applies to all resources on Web site Covers HTML and CSS validation and various other tests Some recursive apps (,rvalidate ) Easily implemented with single line redirect

21. QA Focus – Supporting JISC's 5/99 Programme In-house QA Policies Procedures Audits To date: Web Standards - XHTML, CSS Linking - links to and from Accessibility Usage Statistic Metadata

22. QA Focus – Supporting JISC's 5/99 Programme Documentation and Advice I We have: Surveyed project Web sites and identified areas of lack of compliance with standards and best practices Demonstrated examples of the potential importance of compliance for repurposing resources

23. QA Focus – Supporting JISC's 5/99 Programme Documentation and Advice II In addition we need to provide: Brief focussed advice on the standards Information on how to monitor compliance More Case studies on solutions deployed by projects themselves Guidance on dealing with implementation difficulties and what to do when strict compliance is difficult to achieve Reviews of testing tools

24. QA Focus – Supporting JISC's 5/99 Programme Surveying The Community Surveys of project Web sites have been carried out in order to: Obtain a profile for the community Identify examples of best practices Identify areas in which further advice is needed Surveys included: Accessibility 404 error pages HTML & CSS compliance

25. QA Focus – Supporting JISC's 5/99 Programme Survey Philosophy I The surveys made use of freely-available Web- based tools: Methodology is open No software needs to be installed locally (apart from Web browser) Findings can be reproduced Latest results can be obtained by clicking on link to testing service

26. QA Focus – Supporting JISC's 5/99 Programme Survey Philosophy II The surveys typically examined project entry points and not entire Web site as: This page has the highest profile The aim is to validate a methodology which can be deployed by projects themselves, not to test every page on behalf of the projects

27. QA Focus – Supporting JISC's 5/99 Programme Survey Findings Findings available from http://www.ukoln.ac.uk/qa-focus/surveys/

28. QA Focus – Supporting JISC's 5/99 Programme Providing Motivation There is a need to explain why compliance with standards is important and to provide motivation for projects to update their tools, authoring procedures, etc.This may involve More workshops Evaluating project's QA plans Liaising with projects over the requirements of the key QA procedures applicable to their work Liaising with the Advisory Services over their provision of advice and address maintenance and feedback mechanisms Overseeing QA work

29. QA Focus – Supporting JISC's 5/99 Programme Self Assessment Toolkit We will be developing a self-assessment toolkit for projects to use, by individual projects or across project clusters The toolkit will consist of: Examples of QA procedures Documented examples of use of testing tools Self-assessment questionnaires Advice on standards and best practices Case studies, FAQs etc. http://www.ukoln.ac.uk/qa-focus/toolkit/

30. QA Focus – Supporting JISC's 5/99 Programme QA for Digitisation Do it once…..do it right: Project is fundamentally dependent upon the quality of original product Quality is the pre-requisite to preservation Quality expectations will only grow Delivery problems can be fixed, but capture problems normally can’t

31. QA Focus – Supporting JISC's 5/99 Programme QA For Web Sites The Web is the main delivery mechanism for projects and services An increasing awareness of the importance of accessibility, use of new devices (PDAs, WAP, e-books, …), repurposing of Web content (e.g. archiving) But invalid HTML is still the norm Look to standards – XHTML, XML, CSS, WAI If proprietary formats need to be used, flag them and use in most open way QA Focus is trying to explain why and how

32. QA Focus – Supporting JISC's 5/99 Programme Next Steps Extended Coverage We will be moving on from Web and digitisation to include other areas including: Metadata Multimedia Software development Deployment into service … Next Steps

33. QA Focus – Supporting JISC's 5/99 Programme Next Steps – Metadata Thoughts on QA for metadata: SyntaxCheck syntax on embedded DC Cataloguing RulesEnsure projects have appropriate cataloguing rules InteroperabilityEnsure metadata can interoperate with third parties Fitness for purposeEnsure metadata is appropriate for its purpose Change controlEnsure architecture for managing metadata can cope with change control (cf Exploit Case Study)

34. QA Focus – Supporting JISC's 5/99 Programme Next Steps – Software Thoughts on QA for software: SpecificationProcess for developing systems spec, agreeing spec with stakeholders, etc. MethodologyDocumentation of systems development methodology (cf UML case study) TestingDocumentation of testing methodology DeploymentAwareness of deployment challenges

35. QA Focus – Supporting JISC's 5/99 Programme Limitations There are a number of limitations to the work we have carried out so far: Project Web sites have different purposes (information about the project; communications with project partners; project deliverables themselves; etc.) Projects have different levels of funding, resources, expertise, etc. Projects are at different stages of development (and some have finished)

36. QA Focus – Supporting JISC's 5/99 Programme QA Focus And You! What relevance has QA Focus for you? You can deploy QA Focus practices for your projects, etc You can write case studies and advisory documents: Seen to be working with JISC We can promote your approaches and work This will help instigate best practices within your group You can develop QA procedures in innovative areas (e.g. OAI, thesaurii, ePrints, e-Books, etc.) which QA Focus can use

37. QA Focus – Supporting JISC's 5/99 Programme Questions Any questions?