Presenter : Ching-ting Lin Instructor: Ming-puu Chen Developing a Usability Evaluation Method for E-learning Application: From Functional Usability to Motivation to Learn Zaharias, P. (in press). Developing a usability evaluation method for e-learning application: From functional usability to motivation to learn. Journal of Human Computer Interaction.

Content The need to develop a usability evaluation method Questionnaire Design Methodology Validity analysis Discussion and further work 2

To Develop a Usability Evaluation Method The need to develop a usability evaluation method for e-learning application. –A major challenge of contemporary HCI research is to address user affect (HCI, Human computer interaction) –Traditional usability measures of effectiveness, efficiency and satisfaction are not adequate for new contexts –Motivation to learn is selected in this study –The purpose of this research to develop and empirically test a questionnaire 3

Questionnaire Design Methodology 1) Forming the survey 2) Item sampling 3) Pilot trial 4

Forming the Survey Conceptual framework –A) a combination of web design and instructional design parameters –B) motivation to learn construct (ARCS model) The research focuses on –Asynchronous e-learning applications –Adult learners –For training purposes 5

Pilot Trial 1 (1) The trainees interacted with the e-learning service and courses during a period of four months Participated in usability evaluation was 113 (63 male and 50 female) High internal consistency (range from.707 to.901) The composite variable Motivation to learn shows a very high internal consistency as Alpha coefficient indicates (a=0.926). 6

Pilot Trial 1 (2) The first version contained 54 items representing 12 usability parameters –And 10 items representing to Motivation to Learn construct –a total of 64 items The second version contains 41 items representing 8 usability parameters –And 10 items representing to Motivation to Learn construct –a total of 51 items –High reliability as measured by Cronbach Alpha coefficient (0.961) 7

Pilot Trial 2 Greek language Self-administered the questionnaire 256 respondents (110 male/ 146 female) Version 3 derived –contains 39 items plus 10 items representing to Motivation to Learn, a total of 49 items –The overall alpha for the new version was very high (a=0.934) 8

9

Validity analysis Content validity Criterion-based validity Construct validity 10

Discussion and further work Motivated by the need to address the specificities of e-learning design a usability evaluation method was developed. Need to provide more empirical evidence regarding construct validity Need to explore usabilities attributes and role of affect in synchronous e-learning Use of the proposed questionnaire as a formative evaluation method Provide benchmark information 11