1. Testing And Evaluation

2. Evaluation

3. I’ve Built My Application 1.So you now have a application 2.You’ve followed the guidelines 3.Now its time to evaluate….

4. Why Evaluate? 1.To see if it suits its purpose 2.To see if it is useable 3.To see if the user understands the navigation 4.To see if the interface works 5.To see if it is acceptable for what it was aimed to do

5. Why is usability important? 1.If they can not use it they wont use it 2.If they don’t use it you can loose money 3.If they don’t use it you will loose credibility 4.If the site is an intranet, staff spend to much time plus employers money if unusable 5.Don’t design it for your self design it for the user 6.You should conform to web standards - http://www.useit.com/alertbox/20040913.html http://www.useit.com/alertbox/20040913.html 7.Misconceptions - http://www.useit.com/alertbox/20030908.html

6. How do you evaluate 1.Decide what you are evaluating 2.Decide how you are going to evaluate it 3.Decide who is going to evaluate 4.Perform the evaluation 5.Measure reactions / results 6.Analyse results 7.Report on results

7. Evaluating The Interface 1.Colours 2.Text 3.Use of Icons / Use of metaphor 4.Use of media 5.Does it look and feel right? 6.Design aspects 7.Usability

8. Evaluating Media 1.How many media elements are there 2.Is it appropriate / Is too much media used at once 3.Is it provided by computer or human intervention 4.Can they control it Stop it Pause it Re-run it 5.Can they skip the intro

9. Testing

10. What do we mean by testing? “Testing is the process of trying to discover every conceivable fault or weakness in a work product.” /

11. Why test multimedia? 1.Eliminate errors and bugs 2.Compare against original specification 3.Make Improvements 4.Test usability

12. Testing Methods 1.Black Box Testing 2.White Box Testing 3.Usability Testing How functional is your system? How usable is your system? Functional Specification User Specification

13. Black Box Testing 1.No knowledge of internal workings □No examination of code 2.Inputs and expected outcome 3.Unbiased □Tester and Designer independent 4.Tested from the users point of view 5.Impossible to test every probable stream

14. White Box Testing 1.Explicit knowledge of internal workings required □Only accurate if tester knows what programs must do 2.Knowledge of code used to examine outputs

15. Test Example TaskExpected Outcome Actual Outcome Action Click ‘Home’ Goes to Homepage Goes to Contact page Amend Link Input String in Telephone field Pop message ‘You must enter numbers’ No MessageValidate field Click ‘Exit’Application will exit Application exits No Action

16. How usable is your system? 1.Can a novice user understand it easily? 2.Will I need detailed instructions to learn how to use it? 3.How easy is it to perform a specific task? 4.System needs to be evaluated

17. Example OR Do you want to listen to your message? Oh go on then! Press 1 to listen to your message Press 2 to record a message Press 3...... I don’t have time for this!! Arrrggghhh

18. Evaluation Methods 1.Direct Observation 2.Interviews, Questionnaires and Surveys 3.Heuristics 4.Cognitive Walkthrough

19. Usability and Evaluation Evaluation Evaluation Goals Test Plan Usability Definition Usability Goals Acceptance Criteria User Needs Has Generate Determine Measures Performance Against Structures Identifies

20. Purpose of Usability testing 1.Ensure your application is: □ Effective to use □ Efficient to use □ Safe to use □Have good utility □ Easy to learn □ Easy to remember 2.Consider HCI during development 3.Heuristics Principles

21. Heuristic Principles 1.Consistent placement of interface elements 2.Consistent use of language 3.Consistent, effective and appropriate design 4.Error Prevention 5.Support expert users 6.Useful error messages

22. Benefits of Usability Testing 1.Emphasises usability of your application 2.Creates historical record 3.Reduces services costs 4.Increased probability of sales 5.Improves future goal setting

23. Issues with Usability testing 1.Cost benefit 2.Time consuming 3.Effectiveness of evaluation □Can we predict usability? □How would we know?

24. Cognitive Walkthrough 1.Involves evaluators 2.Walkthrough tasks 3.Note problems 4.Simulate a users problem solving process

25. Cognitive Walkthrough process 1.Characteristics of typical user □User Profile 2.Description of interface 3.Sequence of actions 4.Evaluators walkthrough actions 5.Record critical information 6.Fix problems

26. Why evaluate multimedia? 1.Ensure usability 2.Eliminate errors 3.Comply with principles 4.Attract the correct audience

27. Approaching evaluation 1.To do it correctly we need to follow a defined framework □Identify issues to be measured □Identify methods of measurement □Go through a testing episode □Obtain results □Report on findings

28. Elements to consider 1.Screen Design 2.Menus 3.Colour 4.Metaphor 5.Media Inclusion 6.Task Structuring 7.Navigation

29. Screen design 1.Need a strong visible layout structure 2.Must include titles, and sub- titles 3.Be logical and uncluttered

30. Screen Design (continued) 1.Show well grouped information 2.Be distinguishable related to navigation, interaction, and media 3.Try and observe existing multimedia convention as much as is possible

31. Menus □Must be sensibly located □On screen, or □Drop down □When opened □Are they well laid out and grouped logically □How much screen space do they take up when in use, do hierarchies cause interference problems

32. Colour 1.How well is colour used 2.Do colours contribute to give a design feel or atmosphere to the application 3.Do they enhance the display 4.Is it used to group elements 5.Is colour applied consistently throughout screens 6.Are real world colour conventions observed (especially HCI)

33. Media inclusion 1.What types of media are included 2.Do they combine well 3.Is there a good balance (i.e. interesting) 4.Is control automatic or user controlled 5.Can media elements be selectively manipulated 6.Are media durations appropriate 7.Are special effects applied 8.Are there any technical running problems

34. Task structuring 1.Is the application well defined in terms of tasks and sub-tasks □Logical task sectioning 2.Adequate feedback for errors and completion

35. Navigation 1.Is there an inherent structure to the application to aid the user in building up a mental model 2.Can you work out where you are, where you’ve been, and where to go to easily 3.Is a route recorded to aid backtracking 4.Are visual mapping facilities available

36. Users 1.Does the application fit the intended user type 2.Address background and attributes, and provide functions and facilities the user wants 3.Specifically does the application cater for the correct type of user behaviours: browsers, planners, searchers, explorers, wanderers, the fearful, and fun-seekers

37. Performance 1.Important for both web applications, and CD-ROMs 2.Do pages take too long to load 3.Once loaded, are there running issues related to media, e.g. staggering media clips, losing sound or image, or general delays, giving a feel of poor quality

38. Interaction 1.Typically implicit or explicit interaction is used □Visual objects, or defined buttons 2.Are mechanisms easily identified 3.Do they work? □Can interaction activities easily be worked out 4.If icons are used can □They clearly be identified

39. Fit to Machine Specification 1.Are all the facilities included usable on an average specification PC 2.You yourself will need to identify the best specification for your application 3.May involve testing on several machine set ups and specifications

40. Further Reading 1. Interaction Design beyond human- computer interaction Preece, Rogers & Sharp