2. Measuring Website Usability: Instrument Development, Validation, and Application Big XII IS Research Symposium April 5, 2003 Younghwa “Gabe” Lee University of Colorado at Boulder leey@colorado.edu

3. Agenda  Background  Previous Studies  Research Objective  Research Design  Expected Contribution  Discussion

4. Background  Online business failures are increasing as customers turn away from unusable or unfriendly sites. ‘Build it and they will come’ mentality has led to the demise of e-commerce sites when sites are too late, too buggy, or too complex - Becker and Mottay, 2001  In a poorly designed EC environment, users might be uncomfortable with the uncertainty and ambiguity caused by lack of interaction with websites. -Jahng et al, 2000  The number of shoppers and total sales are still marginal, mainly because of poor interfaces - Jarvenpaa and Todd, 1997

5.  Building a usable website is important since website is the only source for online customers to touch, feel, search, communicate, and experience the products or services available at the online store  Usable websites Build positive attitude (Singh and Dalal, 1999) Increase stickiness (Rettie, 2001) Increase revisit rates (Klein, 1998) Increase online purchase (Palmer, 2002) Increase performance (Nielsen, 2000) Provide more satisfaction (Lund, 1999) Background

6. HCI Company-specific Industry Gurus Few IS researchers Instruments Development Design and Testing How to measure? How to verify? Website Usability HCI MIS SIG HCI Website Usability Research

7. Previous Studies  Measurement problems of website usability No consensus on the definition and dimensions of website usability A number of single-item constructs Intuition and experience-based: Few efforts to develop measurement using scientific methods HCI-oriented objective variables (error rate and download time)  No Investigation of the relationship between website usability constructs No process model

8. Agarwal and Venkatesh (2002) Content Ease of Use Promotion Made-for-the-medium Emotion Palmer (2002) Download Delay Navigability Information Content Interactivity Responsiveness Kim et al. (2002) Firmness Convenience Delight McKinney et al. (2002) Access Usability Entertainment Hyperlinks Navigation Interactivity Zhang and von Dran (2002) Content Enjoyment Privacy User empowerment Visual appearance Technical Support Navigation Credibility Organization

9. Palmer (2002) Download Delay Navigability Information Content Responsiveness Interactivity Perceived Success Download Delay Navigability Information Content Responsiveness Interactivity Perceived Success

10.  Current inconsistency and incompleteness among website usability measurement is the crucial problem of website usability studies.  There is very little in the way of concrete measurement that tells us how good a website really is. Current guidelines, methods, and metrics do help to design better websites, but there is room for improvement - Tarasewich (2000) Motivation of the Study

11. Research Objective  Develop measurement of website usability 18 constructs and 62 instruments have been identified  Investigate the causal relationship between website usability constructs Revealed Causal Mapping approach (Nelson et al., 2000)  Examine the effects of website usability constructs to multiple dependent variables Satisfaction, purchase intention, revisit intention, actual purchase, affect, and loyalty  Investigate generalizability of the new measurement and identify different causal maps under different boundary conditions Gender, Product, Industry and Culture

12. Research Design  Instruments Development Literature review Interviews with web usability experts SUN, IBM, 37 Signals.com website designers A Focus Group Study (IS-majored master-level subjects) A Major survey to 400 Business undergraduate students Exploratory Factor Analysis  Causal Relationship Between Website Usability Constructs Interviewed with experts (n = 20) Interviewed with experienced online customers (n = 40) Data Analysis suggested by Nelson et al. (2000)

13. Research Design  Effects of Website Usability Factors to Diverse Dependent Variables The effects of Price, Time, Scarcity, Convenience, Fun, Usefulness will be examined together CFA and Path Analysis will perform Data Analysis: LISREL  Boundary Conditions Compare websites with different gender-focused Compare websites with different types of products (Hedonic vs Utilitarian) Compare websites with different cultures (U.S. vs Japan) Compare websites with different stakeholders (Customers, Designers, and Managers) Data Analysis: PLS

14. Flexibility Tele- Presence Community Scanability Learnability Readability Interactivity Consistency Simplicity Navigability Timeliness Content Relevancy Scope Privacy Reliability/ Accessibility Security Credibility Website Usability Constructs

15. Flexibility Tele- Presence Community Scanability Learnability Readability Interactivity Consistency Simplicity Navigability Timeliness Content Relevancy Scope Privacy Reliability/ Accessibility Security Credibility + + - + - + - + Causal Relationship between Website Usability Constructs

16. Satisfaction Purchase Intention Affect Loyalty Actual Purchase Revisit Intention Flexibility Tele- Presence Community Scanability Learnability Readability Interactivity Consistency Simplicity Navigability Timeliness Content Relevancy Scope Privacy Reliability/ Accessibility Security Credibility + + - + - + - + Effects of Usability Constructs to Diverse Dependent Variables Price Time Scarcity Convenience Fun Usefulness

17. Expected Contribution  Develop new measurement of website usability provide a better means to evaluate website design quality  Identify causal relationship between website usability constructs Provide a deeper understanding of how online customers build their usability perception  Examine effects of website design factors to diverse dependent variables justify the investment on website usability  Investigate measurement’s generalizability and causal maps under different boundary conditions help to perform future study using the measurement help designers to allocate limited resources to the most important usability factors

18. Discussion

20. Revealed Causal Mapping Repertory Grid Causal Relationship Identification Construct Identification Guided Interviews Open Interviews No requirement for multiple elements Multiple elements are required Identify constructs and causal Identify constructs using Relationship based on repeated Triadic Methods to identify Interviews for identifying Mid-theory exists No requirement for mid-theory

21. Discussion  Alternative ways of conducting the research  Process Model (e.g TAM) vs Multiple Dependent Variables  Boundary conditions valuable to be observed