Testing And Evaluation

Evaluation

I’ve Built My Application 1.So you now have a application 2.You’ve followed the guidelines 3.Now its time to evaluate….

Why Evaluate? 1.To see if it suits its purpose 2.To see if it is useable 3.To see if the user understands the navigation 4.To see if the interface works 5.To see if it is acceptable for what it was aimed to do

Why is usability important? 1.If they can not use it they wont use it 2.If they don’t use it you can loose money 3.If they don’t use it you will loose credibility 4.If the site is an intranet, staff spend to much time plus employers money if unusable 5.Don’t design it for your self design it for the user 6.You should conform to web standards Misconceptions -

How do you evaluate 1.Decide what you are evaluating 2.Decide how you are going to evaluate it 3.Decide who is going to evaluate 4.Perform the evaluation 5.Measure reactions / results 6.Analyse results 7.Report on results

Evaluating The Interface 1.Colours 2.Text 3.Use of Icons / Use of metaphor 4.Use of media 5.Does it look and feel right? 6.Design aspects 7.Usability

Evaluating Media 1.How many media elements are there 2.Is it appropriate / Is too much media used at once 3.Is it provided by computer or human intervention 4.Can they control it Stop it Pause it Re-run it 5.Can they skip the intro

Testing

What do we mean by testing? “Testing is the process of trying to discover every conceivable fault or weakness in a work product.” /

Why test multimedia? 1.Eliminate errors and bugs 2.Compare against original specification 3.Make Improvements 4.Test usability

Testing Methods 1.Black Box Testing 2.White Box Testing 3.Usability Testing How functional is your system? How usable is your system? Functional Specification User Specification

Black Box Testing 1.No knowledge of internal workings □No examination of code 2.Inputs and expected outcome 3.Unbiased □Tester and Designer independent 4.Tested from the users point of view 5.Impossible to test every probable stream

White Box Testing 1.Explicit knowledge of internal workings required □Only accurate if tester knows what programs must do 2.Knowledge of code used to examine outputs

Test Example TaskExpected Outcome Actual Outcome Action Click ‘Home’ Goes to Homepage Goes to Contact page Amend Link Input String in Telephone field Pop message ‘You must enter numbers’ No MessageValidate field Click ‘Exit’Application will exit Application exits No Action

How usable is your system? 1.Can a novice user understand it easily? 2.Will I need detailed instructions to learn how to use it? 3.How easy is it to perform a specific task? 4.System needs to be evaluated

Example OR Do you want to listen to your message? Oh go on then! Press 1 to listen to your message Press 2 to record a message Press I don’t have time for this!! Arrrggghhh

Evaluation Methods 1.Direct Observation 2.Interviews, Questionnaires and Surveys 3.Heuristics 4.Cognitive Walkthrough

Usability and Evaluation Evaluation Evaluation Goals Test Plan Usability Definition Usability Goals Acceptance Criteria User Needs Has Generate Determine Measures Performance Against Structures Identifies

Purpose of Usability testing 1.Ensure your application is: □ Effective to use □ Efficient to use □ Safe to use □Have good utility □ Easy to learn □ Easy to remember 2.Consider HCI during development 3.Heuristics Principles

Heuristic Principles 1.Consistent placement of interface elements 2.Consistent use of language 3.Consistent, effective and appropriate design 4.Error Prevention 5.Support expert users 6.Useful error messages

Benefits of Usability Testing 1.Emphasises usability of your application 2.Creates historical record 3.Reduces services costs 4.Increased probability of sales 5.Improves future goal setting

Issues with Usability testing 1.Cost benefit 2.Time consuming 3.Effectiveness of evaluation □Can we predict usability? □How would we know?

Cognitive Walkthrough 1.Involves evaluators 2.Walkthrough tasks 3.Note problems 4.Simulate a users problem solving process

Cognitive Walkthrough process 1.Characteristics of typical user □User Profile 2.Description of interface 3.Sequence of actions 4.Evaluators walkthrough actions 5.Record critical information 6.Fix problems

Why evaluate multimedia? 1.Ensure usability 2.Eliminate errors 3.Comply with principles 4.Attract the correct audience

Approaching evaluation 1.To do it correctly we need to follow a defined framework □Identify issues to be measured □Identify methods of measurement □Go through a testing episode □Obtain results □Report on findings

Elements to consider 1.Screen Design 2.Menus 3.Colour 4.Metaphor 5.Media Inclusion 6.Task Structuring 7.Navigation

Screen design 1.Need a strong visible layout structure 2.Must include titles, and sub- titles 3.Be logical and uncluttered

Screen Design (continued) 1.Show well grouped information 2.Be distinguishable related to navigation, interaction, and media 3.Try and observe existing multimedia convention as much as is possible

Menus □Must be sensibly located □On screen, or □Drop down □When opened □Are they well laid out and grouped logically □How much screen space do they take up when in use, do hierarchies cause interference problems

Colour 1.How well is colour used 2.Do colours contribute to give a design feel or atmosphere to the application 3.Do they enhance the display 4.Is it used to group elements 5.Is colour applied consistently throughout screens 6.Are real world colour conventions observed (especially HCI)

Media inclusion 1.What types of media are included 2.Do they combine well 3.Is there a good balance (i.e. interesting) 4.Is control automatic or user controlled 5.Can media elements be selectively manipulated 6.Are media durations appropriate 7.Are special effects applied 8.Are there any technical running problems

Task structuring 1.Is the application well defined in terms of tasks and sub-tasks □Logical task sectioning 2.Adequate feedback for errors and completion

Navigation 1.Is there an inherent structure to the application to aid the user in building up a mental model 2.Can you work out where you are, where you’ve been, and where to go to easily 3.Is a route recorded to aid backtracking 4.Are visual mapping facilities available

Users 1.Does the application fit the intended user type 2.Address background and attributes, and provide functions and facilities the user wants 3.Specifically does the application cater for the correct type of user behaviours: browsers, planners, searchers, explorers, wanderers, the fearful, and fun-seekers

Performance 1.Important for both web applications, and CD-ROMs 2.Do pages take too long to load 3.Once loaded, are there running issues related to media, e.g. staggering media clips, losing sound or image, or general delays, giving a feel of poor quality

Interaction 1.Typically implicit or explicit interaction is used □Visual objects, or defined buttons 2.Are mechanisms easily identified 3.Do they work? □Can interaction activities easily be worked out 4.If icons are used can □They clearly be identified

Fit to Machine Specification 1.Are all the facilities included usable on an average specification PC 2.You yourself will need to identify the best specification for your application 3.May involve testing on several machine set ups and specifications

Further Reading 1. Interaction Design beyond human- computer interaction Preece, Rogers & Sharp