Acceptance of TEL: Key Success Factors and Reasons of Failure PPE Summer School 2008, Ohrid, FYROM Patrick Johnscher (TU Darmstadt) Sebastian Kelle (Open University, Heerlen) Steinn E. Sigurðarson (WU Vienna) Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Training & Learning Training measures can be used simply because there is no alternative… ….but it should be about good design! 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 2 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Good TEL Design … is more than just a collection of PowerPoint slides… 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 3 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

So why don't we design what the users want? 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 4 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Why is acceptance of TEL important? What we just learned from Dilbert: TEL design should consider the users' requirements A system that meets expectations is more likely to gain acceptance and get used – at least if there are learning alternatives Common misperception: "If we build it, they will come!" E-Learning is not a field of dreams: just providing technology doesn't mean learners will use the systems At the end of the day, TEL is no end in itself: there has to be a measurable learning outcome, and a business case has to be made Davis (1989): "Technology Acceptance" Information technology offers the potential for substantially improving white collar performance But performance gains are often obstructed by users' unwillingness to accept and use available systems Because of the persistence and importance of this problem, explaining user acceptance has been a long-standing issue in MIS research 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 5 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

A What-went-wrong Example: COVCELL COVCELL Project – Effective language learning (http://www.covcell.org) Most important factor for language learning: “learning through conversation” 88% of students 75.9% of teachers So, interaction and communication is quite important? COVCELL developed tools such as: User Presence and Chat Audio/Video Conferencing Collaborative Whiteboard Audio Recording User acceptance as a basis for design requirement specifications 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 6 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 6 18. Juni 2008 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | | | 6 | 6

COVCELL: Percentage of users happy with… 14,3% 7,1% 28,6% 71,4% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% …audio recording …whiteboard …audio/video conferencing …user presence and chat …using the wiki …using the forum 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 7 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 7 18. Juni 2008 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | | | 7 | 7

COVCELL: User Ratings of… 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 …audio recording …whiteboard …audio/video chat ...user presence and chat …wiki …forum Language Learning General Scale: 0-5 4.2 / 5 4.5 / 5 3.6 / 5 3.4 / 5 2.3 / 5 2.4 / 5 3.2 / 5 3.0 / 5 2.8 / 5 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 8 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 8 18. Juni 2008 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | | | 8 | 8

COVCELL: People didn't use… …user presence and chat 85,7% …audio/video conferencing 78,6% …whiteboard 92,9% …audio recording 71,4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 9 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 9 18. Juni 2008 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | | 9

A Few Questions… What is "acceptance"? Let's try some definition… Which factors influence acceptance? A closer look at acceptance models… 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 10 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Acceptance – A Definition "Acceptance of an innovation or technology describes the positive adoption decision of users – in contrast to the rejection of an innovation or technology." (Simon 2001, p. 87) Technology acceptance usually consists of two separate aspects: An attitude component, comprising affective and rational aspects A behavioral component, describing the adoption of an innovation in the form of actual observable behavior (e.g. use of a system) Since the behavioral component is usually easy to observe (and thus easy to measure), modeling and research is mostly focused on the attitude component of user acceptance: Affective aspects, including motivational and emotional attributes Rational aspects, including cost-benefit relations as seen from the users' individual perspective 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 11 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Acceptance Models Frequently cited models in the literature are: Technology Acceptance Model Technology Acceptance Model 2 The Unified Theory of Acceptance and Use of Technology Basic concept underlying user acceptance models: 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 12 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Technology Acceptance Model (TAM) Developed by Davis (1989) Main idea: Perceived usefulness and perceived ease of use are fundamental factors influencing the user acceptance as they influence the user’s attitude towards the system. Davis’ TAM model 1993 proposes that perceived usefulness and perceived ease of use are fundamental factors influencing the user acceptance as they influence the user’s attitude towards the system. He defined perceived usefulness as "the degree to which a person believes that using a particular system would enhance his or her job performance" and perceived ease of use as "the degree to which a person believes that using a particular system would be free from effort" (Davis, 1989). 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 13 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Technology Acceptance Model (TAM2) Developed by Venkatesh and Davis (2000) Extension of the original TAM model to explain perceived usefulness and usage intentions in terms of social influence process and cognitive instrumental processes. In TAM2 the social influence process highlights the impact of three inter-related social forces impinging on an individual facing the opportunity to adopt or reject a new system: (a) subjective norm, defined as a “person’s perception that most people who are important to him think he should or should not perform behavior in question” (b) voluntariness and (c) image factor for user acceptance. In cognitive instrumental process, the TAM 2 highlights the individual’s job relevance and output quality. Results demonstrability and perceived ease of use are other fundamental determiners of user acceptance. 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 14 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Unified Theory of Acceptance and Use of Technology (UTAUT) Developed by Venkatesh et al (2003) Consolidation of eight prominent technology user acceptance models. The eight models reviewed are: Theory of Reasoned Action (TRA) Technology Acceptance Model (TAM) Motivational Model (MM) Theory of Planned Behavior (TPB) Combined TAM and TPB (C-TAM-TPB) Model of PC Utilization (MPCU) Innovation Diffusion Theory (IDT) Social Cognitive Theory (SCT) The UTAUT states that four constructs play a significant role as direct determinants of user acceptance and usage behavior. They are: performance expectancy effort expectancy social influence facilitating conditions The UTAUT theory presented by Venkatesh et al (2003) states that four constructs play a significant role as direct determinants of user acceptance and usage behavior. They are: performance expectancy, effort expectancy, social influence, and facilitating conditions. The authors defined the performance expectancy as the degree to which an individual believes that using the system will help him or her to attain gains in job performance. The effort expectancy is defined as the degree of ease associated with the use of the system. The social influence is defined as the degree to which an individual perceives that important others believe he or she should use the new system. Facilitating conditions are defined as the degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system (Venkatesh et al, 2003). 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 15 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Unified Theory of Acceptance and Use of Technology (UTAUT) The UTAUT theory presented by Venkatesh et al (2003) states that four constructs play a significant role as direct determinants of user acceptance and usage behavior. They are: performance expectancy, effort expectancy, social influence, and facilitating conditions. The authors defined the performance expectancy as the degree to which an individual believes that using the system will help him or her to attain gains in job performance. The effort expectancy is defined as the degree of ease associated with the use of the system. The social influence is defined as the degree to which an individual perceives that important others believe he or she should use the new system. Facilitating conditions are defined as the degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system (Venkatesh et al, 2003). 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 16 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Acceptance Model for TEL? 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 17 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Influence Factors (Exogenous Variables): Draft Classification Technological Organizational Didactical / Pedagogical Personal Cognitive Cultural Motivational … Others? 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 18 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Now over to you… Form up in groups of 4-5 students Think about factors influencing the acceptance of TEL – and reasons why TEL implementation fails Try to cluster the success factors / reasons of failure into categories, e.g. "organizational", "technological" Identify connections and dependencies between the separate factors: which factors directly influence acceptance; which influence each other (indirect influence on acceptance)? Visualize your group results on a flipchart 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 19 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" 19

Literature Venkatesh, V., Morris, M.G., Davis, F.D., and Davis, G.B.: "User Acceptance of Information Technology: Toward a Unified View," MIS Quarterly, 27, 2003, 425-478. Venkatesh, V. and Davis, F.D.: "A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies," Management Science, 46, 2000, 186-204. Davis, F. D., Bagozzi, R. P., and Warshaw, P. R.: "User Acceptance of Computer Technology: A Comparison of Two Theoretical Models," Management Science, 35, 1989, 982-1003. Goodhue, D. L.; Thompson, R. L. (1995): "Task-Technology Fit And Individual Performance", MIS Quarterly, Vol. 19, Issue 2, 213-237. Rogers, E. M. (2003): Diffusion of innovations (5th ed.). New York: Free Press. Nanayakkara, C. (2007): "A Model of User Acceptance of Learning Management Systems: a study within Tertiary Institutions in New Zealand", Educause Australasia 2007, http://www.caudit.edu.au/educauseaustralasia07/authors_papers/Nanayakkara-361.pdf. Tai, L. (2008): Corporate E-Learning: An Inside View of IBM's Solutions. Oxford University Press, New York. 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 20 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Construct Definitions Attitude: Individual's positive or negative feeling about performing the target behavior (e.g., using a system). Behavioral intention: The degree to which a person has formulated conscious plans to perform or not perform some specified future behavior. Computer anxiety: The degree of an individual’s apprehension, or even fear, when she/he is faced with the possibility of using computers. Computer playfulness: The degree of cognitive spontaneity in microcomputer interactions. Computer self-efficacy: The degree to which an individual beliefs that he or she has the ability to perform specific task/job using computer. Effort expectancy: The degree of ease associated with the use of the system. Facilitating conditions: The degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system. 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 21 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Construct Definitions Image: The degree to which use of an innovation is perceived to enhance one's status in one's social system. Job relevance: Individual's perception regarding the degree to which the target system is relevant to his or her job. Objective usability: A comparison of systems based on the actual level (rather than perceptions) of effort required to complete specific tasks. Output quality: The degree to which an individual believes that the system performs his or her job tasks well. Performance expectancy: The degree to which an individual believes that using the system will help him or her to attain gains in job performance. Perceived ease of use: See the definition of effort expectancy. Perceived enjoyment: The extent to which the activity of using a specific system is perceived to be enjoyable in it’s own right, aside from any performance consequences resulting from system use. Perceived usefulness: See the definition of performance expectancy. 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 22 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Construct Definitions Perception of external control: See the definition of facilitating conditions. Result demonstrability: Tangibility of the results of using the innovation. Social influence: The degree to which an individual perceives that important others believe he or she should use the new system. Subjective norm: Person's perception that most people who are important to him think he should or should not perform the behavior in question. Voluntariness: The extent to which potential adopters perceive the adoption decision to be non-mandatory. 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 23 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 24 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Acceptance – A Definition "Acceptance of an innovation or technology describes the positive adoption decision of users – in contrast to the rejection of an innovation or technology." (Simon 2001, p. 87) "Accordingly, the innovation-decision process is the process through which an individual or other decision-making unit passes from first knowledge of an innovation, to forming an attitude toward the innovation, to a decision to adopt or reject, to implementation of the new idea, and to confirmation of this decision." (Rogers: Diffusion of Innovations, 2003, p. 161) 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 25 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Acceptance – A Definition Technology acceptance usually consists of two separate aspects: An attitude component, comprising affective and rational aspects A behavioral component, describing the adoption of an innovation in the form of actual observable behavior (e.g. use of a system) Since the behavioral component is usually easy to observe (and thus easy to measure), modeling and research is mostly focused on the attitude component of user acceptance: Affective aspects, including motivational and emotional attributes Rational aspects, including cost-benefit relations as seen from the users' individual perspective 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 26 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

A Few Questions… What is "acceptance"? Let's try some definition… Which factors influence acceptance? A closer look at acceptance models… How can I measure acceptance? Using acceptance models in empirical research… 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 27 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"

Using Acceptance Models in Empirical Research Models are so-called "Structural Equation Models" (SEM) or "Causal Models" SEM encourages confirmatory rather than exploratory modeling; thus, it is suited to theory testing rather than theory development. There are usually two main parts to SEM: the structural model showing potential causal dependencies between endogenous and exogenous variables, and the measurement model showing the relations between the latent variables and their indicators. Benefits: User Acceptance Models like TAM, UTAUT and others are widely accepted and represent state-of-the-art in IS research Models come with hypotheses "on board"; you don't have to formulate your own Scales and question items exist and have been tested in a number of previous studies ("tried and true") Drawbacks: Relatively high demands on data quality Parameter estimation using covariance-based techniques can be considered advanced statistics 2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 28 Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure"