1. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY A Multidisciplinary tool for the evaluation of usability, pedagogical usability, accessibility and informational quality of web-based courses Kirsi Silius, Anne-Maritta Tervakari & Seppo Pohjolainen Tampere University of Technology (TUT) Finland PEG2003 St.Petersburg, Russian

2. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 2 The main phases of survey at TUT –Context analysis: to find significant background factors, which have strong impact to the instructional value of the certain web-based learning environment. –Advantages and disadvantages of web-based learning in various contexts. What are the critical factors of quality in web-based learning? –Assessment of implementation of web courses using multidisciplinary framework based on usability, pedagogical usability, accessibility and informational quality –Value added of web-based courses in various contexts.

3. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 3 Viewpoints of evaluation usability accessibility presentation of information presentation of information reliability of information reliability of information pedagogical usability pedagogical usability value added user group quality of information fluency of use instructional quality benefits

4. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 4 Usefulness of web-based learning environments based on Nielsen (1993), modified by Tervakari & Silius 2002-03 Utility Easy to learn Efficient to use Usability Easy to remember Few errors Subjectively pleasing Value added Support for organisation of the teaching process Support for learning, tutoring and scaffolding process and support for the achievement of learning objectives Support for the development of learning skills (interaction with other actors and growth of learners` autonomy Organisation of teaching process Development of quality of teaching Development of learners’ learning skills Testing and development of educational ICT Pedagogical usability Usefulness Social acceptability System acceptability Practical acceptability Cost Compatibility Reliability Etc. Nielsen, J. 1993. Usability Engineerings. Boston (MS.): Academic press.

5. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 5 Evaluation sections

6. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 6 Purpose of this evaluation tool This tool is not –for making ranking lists of web-based courses –for evaluate the validity or substantiveness of the content This tool is to –help teachers to pay attention to properties of their web pages which might need some improvement

7. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 7 Start of evaluation 1. 2. 3.

8. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 8 Choose sections

9. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 9 Questions to be answered

10. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 10 Report 1: Results of the evaluation

11. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 11 Report 2: Guides and tips

12. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 12 Report 3: Own answers

13. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 13 Filter out unrelevant questions

14. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 14 Browse through guides and tips

15. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 15 Guides and tips

16. DIGITAL MEDIA INSTITUTE hypermedia laboratory Finnish Virtual University TAMPERE UNIVERSITY OF TECHNOLOGY 16 Futher development More criteria and guidelines into pedagogical usability section More references into all sections Questionnaires for real users The tool will help to match data collected from evaluators and real users