1. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 1 CSAW 2004 Towards Appraising Online Stores Ernest Cachia Mark Micallef Software Engineering Process Improvement Research Group (SEPI)

2. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 2 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Introduction

3. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 3 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Overall Goals

4. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 4 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 The First Step

5. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 5 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Initial Targets Identify most important quality attributes in e- commerce systems Limit Scope to the Online Store variety of E- Commerce Systems (for the time being) Success of such systems depends largely on users accessing a website and buying products or services Therefore, users’ opinions should be given a large weighting

6. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 6 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Overview of Activities 1.Research into existing work 2.Discovering how e-commerce systems differ from generic systems 3.Designing a questionnaire 4.Conduct a survey amongst 350 e- commerce users 5.Analysis of results 6.Extraction of important e-commerce attributes (and their importance ranking)

7. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 7 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 How do E-Commerce Systems Differ?

8. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 8 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Why was this information required? Identifying why e-commerce systems differ was an important step: –Further justifies the need for an e-commerce metrication framework –Allows us to identify quality attributes based on these differences 5 main differences were identified

9. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 9 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 How E-Commerce System Differ Content Driven (Navigability) Exposed to the world (Security) Browser-Based (Portability) Enormous User Base (Scalability, Multilinguality) Frequent Change (Maintainability, Reliability)

10. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 10 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Survey Organisation

11. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 11 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Survey Design Designed to rank importance of quality attributes defined above Divided into two parts: –Section 1: Collection of demographic information about users –Section 2: Collection of information about which attributes are deemed most important by users

12. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 12 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Example Questions from Section 1 Age Group Main use of e-commerce (personal or business) Education Level Frequency of online shopping Products/Services bought online OS, Browser, Devices

13. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 13 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Example Questions from Section 2 Abandoned Transactions Reasons for abandoned transactions “First Impression” Factors Intentions when visiting an e-commerce site Ranking of attributes/features

14. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 14 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Survey Results

15. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 15 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Our Sample Space 350 Respondents 94% of participants were under 50 years old Browser usage, OS usage and Device usage compare to other surveys carried out

16. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 16 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Browsers

17. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 17 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Operating Systems

18. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 18 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Devices

19. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 19 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Most Popular Products

20. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 20 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Abandoned Transactions

21. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 21 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Reasons for Abandoned Transactions

22. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 22 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Security and Privacy Issues (1/3) Security turned out to be an important factor For example, when asked to identify a reason why they would prefer not to use e-commerce, users replied as follows…

23. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 23 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Security and Privacy Issues (2/3) How sure would you have to be about a site’s security capabilities before you buy from it?

24. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 24 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Security and Privacy Issues (3/3) When asked to rank the importance of quality attributes in order of importance, security received and average scored of 6.24 (out of a possible 7). This made it the most important quality attribute as perceived by users of e- commerce systems We recommended security as an essential quality attribute in e-commerce systems

25. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 25 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Portability Issues (1/4) Refers to the extent to which a system is accessible from different operating systems, browser and devices Participants ranked portability as the 5 th most important attribute (out of 7) Case for portability not too strong if one relies on the results of this survey

26. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 26 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Portability Issues (2/4) 98% of respondents use Windows TM -based OSs Almost 93% use Desktop PCs 95% use Internet Explorer TM Less than 4% of users who abandoned transactions did so because of compatibility problems

27. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 27 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Portability Issues (3/4) Results seem to indicate that portability is not important However one must be careful: –88.7% of users will not change browsers so as to be able to view a particular site –A small percentage of customer lost due to lack portability might still amount to large amounts of lost revenue –New mobile technologies are likely increase the importance of portability issues in e-commerce

28. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 28 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Portability (4/4) Portability is recommended as a necessary attribute in e-commerce systems However, in the current environment, compatibility with dominant technologies would seem to ensure a sufficient reach

29. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 29 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Performance and Scalability (1/3) Speed is important to users 33% of users who abandoned transactions mid-way through did so because the site was too slow Only 18.4% would remain unconditionally loyal to their favorite site if it became slow

30. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 30 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Performance and Scalability (2/3) What is the most important “first impression” factor you look for in a site?

31. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 31 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Performance and Scalability (3/3) Participants ranked performance as the 4 th most important quality attribute We are recommending Performance as an essential quality attribute

32. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 32 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Navigability (1/3) Why did you abandon transactions mid-way through? Also more likely to simply find another site

33. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 33 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Navigability (2/3) 72% of users know beforehand what they are looking for when visiting an online store This indicates that search and navigation features are important 30% of participants also chose navigation as their primary “first impression” factor

34. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 34 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Navigability (3/3) Navigability was ranked as the 3 rd most important attribute by users We are recommending usability with an emphasis on navigability as an essential quality attribute in online stores

35. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 35 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Multi-Lingual Features Not being recommended as an essential attribute The importance of such features is highly dependant on the context in which an online store is operating 51% of participants claim e-commerce systems could still be classified as a quality system if it had no multi-lingual features Respondents ranked this feature in 6 th place (out of 7)

36. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 36 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Low-Bandwidth Version of Site Some e-commerce sites have a text-only or low- bandwidth version available Useful for users with slow connections 52.5% of participants deem such a feature unnecessary Users ranked this attribute as the least important Considering increasing popularity of broadband connections, we are not recommending this as a necessary attribute

37. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 37 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Ranking of Essential Attributes 1.Security 2.Reliability 3.Navigability 4.Performance 5.Portability

38. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 38 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Conclusions and Further Work

39. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 39 Introduction  First Step  How E-Commerce Systems Differ  Survey Design  Results  Conclusions and Future Work CSAW 2004 Conclusions and Further Work These results are an important first step to our overall goals Since this research we have explored: –Information Architecture –Graph Theory –Security Further work will include: –Investigation into each of the above attributes and how they can be measured –Development of a metrication framework that will combine metrics for different attributes together –Development of a list of guidelines –Incorporation of guidelines into framework

40. Towards Appraising Online Stores SEPI Research Group, Department of Computer Science and AI University of Malta 40 CSAW 2004 Questions and Discussion