Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Interactive Systems Design MSc in Communications, Computing and Human-Centred Systems Dr. Chris Baber.

Similar presentations

Presentation on theme: "1 Interactive Systems Design MSc in Communications, Computing and Human-Centred Systems Dr. Chris Baber."— Presentation transcript:

1 1 Interactive Systems Design MSc in Communications, Computing and Human-Centred Systems Dr. Chris Baber

2 2 Human-Centred Systems Dr Chris Baber Room: N313 Tel. ext.: 43965 Email:

3 3 Aims Introduce some basic concepts from Cognitive Psychology Demonstrate relationships between these concepts and interactive systems design

4 4 Objectives After completing this module, students will be able to:  Critique user interfaces through the application of appropriate concepts from cognitive psychology  Offer guidelines on how to improve a design

5 5 Assignment 1. Select ANY mobile telephone user interface 2. Critically evaluate the user interface in terms of: 1. layout 2. use of colour 3. Gestalt 4. consistency 5. support for navigation 3. Explain how and why the user interface might present problems to users 4. Consider the (non-computer) metaphor(s) that can be related to the design 5. Explain what assumptions you believe that the designers have made in terms of the way that users interpret the user interface 6. Produce a redesign of the user interface that you believe is superior to the current version and e xplain why your redesign is better

6 6 Marking Scheme 1. Presentation of system image and screen-shots (images and description) - up to 10 marks 2. Definition of a ‘good’ user interface - up to 10 marks 3. Discussion of key topics from cognitive psychology - up to 30 marks 4. Critique of selected user interface, in terms of 2 and 3 - up to 25 marks 5. Redesign (images and description) - up to 10 marks 6. Why is your redesign superior to the current design? - up to 10 marks Up to 5 additional marks for presentation of report

7 7 Useful References Ji, Y.G. et al., 2006, A usability checklist for the usability evaluation of mobile phone user interface, International Journal of Human-Computer Interaction, 20 (3), pp. 207-231 Bocker, M. and Suwita, A., 1999, The evaluation of the Siemens C10 mobile phone – usability testing beyond ‘quick and dirty’, Proceedings of the 43 rd Annual Meeting of the Human Factors and Ergonomics Society, Santa Monica, CA: HFES, pp.379-383 Ziefle, M., 2002, Usability of menu structures and navigation keys in mobile phones: a comparison of the ease of use in three different brands, Proceedings of the 6 th International Scientific Conference on Work with Display Units, Berlin, pp.359- 361 Jokela, T. et al., 2006, Methods for Quantitative Usability requirements: a case study on the development of the user interface of a mobile phone, Personal and Ubiquitous Computing, 10 (6), 345-355

8 8 Timetable Monday 29thTuesday 30 th Wednesday 31st 10 - 111. Introduction to Module 6. Navigation in user interfaces 10. Paper prototyping 11 - 122. Fundamentals of graphical user interface design 7. Interaction Devices: entering data 11. Distributed Cognition 12 - 13. Link and Layout Analysis 8. Interaction Devices: Selecting objects 12. Collaborative Work 2 - 34. Use of Colour and Highlighting 9. Multimodal HCI13. Future HCI 4 - 55. Commands and Menus 10. Mobile HCI14. Future HCI

9 9 1. Introduction

10 10 Design Principles 1. Know your user 2. Build on systems that are familiar to your user 3. Display available functions 4. Ensure Coherence and Consistency 5. Visualise states and transitions 6. Design shortcuts 7. Design Help 8. Tolerate user error 9. Provide Context 10. Manage Focus 11. Display Grammar

11 11 Varieties of Technology Technology mediates human action on the world in ways that… Augment Correct Extend Modify & Disrupt …human performance and ability

12 12 Augment / Correct / Extend Technologies ‘augment’ human abilities SUBSTITUTION: calculators REPRESENTATION: shopping lists Technologies ‘extend’ human abilities CORRECTION: spectacles, hearing aids increasing RANGE: telescope, telephone

13 13 Modify Technology changes:  Task sequence  Required actions  Information content Example:  Calculate 9 hours of work at £3.50 per hour. What if you add another hour?

14 14 Disrupt Changes ways of working  Requires new ways of thinking or acting Interferes with activities  MicroSoft PaperClip  PDA in meetings  Mobile phones in lectures

15 15 How different ‘models of the user’ lead to different assumptions on how to Minimise Human Error Explain errorReduce error Idiot Design faultMake as simple as possible Designer Error freeNo need Computer Follow instructionClarify instruction Information Processor BreakdownModes; forcing functions Active Interpreter Not CompatibleDesign for mental model

16 16 ‘Instant Experts’ You get a new mobile telephone for Xmas, do you: 1. Read the manual from cover to cover and then switch on the phone? 2. Switch on the phone and try to use it? 3. Switch on the phone and read the manual when you have problems?

17 17 Using tools Physical appearance Knowledge of use Sequence of activity

18 18 Affordance See handle Reach out hand Grasp handle Turn handle Pull door

19 19 The Cooker Problem #1 Which control acts on which ring?

20 20 The Cooker Problem #2 Which control acts on which ring?

21 21 Paul Fitts: Spatial Compatibility

22 22 Direction of motion Stereotypes ad b c ?? Clockwise = increase Clockwise = right Clockwise = away from control Clockwise = increase on scale 12345671234567 12345671234567

23 23 Clockwise to Increase? 12345671234567

24 24 Conclusions… We have learned ‘routines’ for how to use many sorts of technology We apply these routines ‘automatically’ When the routines succeed, they are reinforced When the routines fail, we think about what we’re doing

25 25 Expectations and Affordances People “know” how technology works…  ‘Out of box’ experience  Reading instructions  Prior knowledge and expectations

26 26 Automaticity Norman and Shallice (1980) Fully automatic processing controlled by SCHEMATA Partially automatic processing controlled by either Contention Scheduling Supervisory Attentional System (SAS)

27 27 Supervisory Attentional System Model Perceptual System Supervisory Attentional System Effector System Contention scheduling Trigger database Control schema

28 28 Contention Scheduling Gear changing when driving involves many routine activities but is performed ‘automatically’ – without conscious awareness When routines clash, relative importance is used to determine which to perform – Contention Scheduling e.g., right foot on brake or clutch

29 29 SAS activation Driving on roundabouts in France  Inhibit ‘look right’; Activate ‘look left’  SAS to over-ride habitual actions SAS active when: Danger, Choice of response, Novelty etc.

30 30 Know your user… When designing any form of user interface, you should ask:  What am I trying to help the reader (or user) of this display do?  How does the information I am presenting to them help with their thinking or their activity?

31 31 Seven Stage Action Model [Norman, 1990] GOAL OF PERSON

32 32 Routes from Intention to Action Route 1:Automaticity  Apply known routine  Compatibility  Affordance Route 2: Formulate plan  Problem solving  Situated actions

33 33 Problem Solving A problem is something that doesn’t solve easily A problem doesn’t solve easily because:  you don’t have the necessary knowledge or,  you have misrepresented part of the problem If at first you don’t succeed, try something else Tackle one part of the problem and other parts may fall into place

34 34 Problem Solving Representation affects strategy More than one possible solution Solution limited by boundary conditions Active involvement and testing

35 35 Problem SEND + MORE MONEY

36 36 Functional Fixedness Strategy developed in one version of the problem Strategy might be inefficient XIV ÷ X vs. XXXX ÷ X Convert numerals or just ‘see’ 4

37 37 Interacting with Products as Problem Solving User has a ‘goal’ to achieve using the product The product offers functions to the user The user must select the ‘best’ function to progress towards the goal The interaction can be thought of as movement through a space of possible actions towards a goal Selection of the ‘best’ action will be influenced by available functions and their interpretation

38 38 Situated Actions ‘Plans’ imply a specified sequence of actions…  plans are representations of action …but some actions are ‘opportunistic’, i.e., supported by the current context-of-use Suchman (1992) terms these ‘situated actions’  plans are resources for action

39 39 2. Fundamental of graphical user interface design

40 40 Information Design can Influence Thinking John Snow, 1854m Mapping death from cholera, in London, showed clusters around water pump (and not near brewery)

41 41 Attributes affecting utility Readability Words Syntax Legibility Font design Contrast Illumination Luminance Conspicuity Graphic design Spatial coding Temporal coding Shape coding Colour coding Display Psychological process Extraction of meaning Comprehension Mediating User attributes

42 42 Gestalt Principles

43 43 Density and Clutter Density is related to available screen space  E.g 80 x 24 line display = 1920 character spaces The proportion of filled spaces = Density Density averages 25% but rarely exceeds 50%

44 44 Levels of Density 70% density 50% density 30% density Shneiderman, 1992

45 45 Reducing Density Grouping / tabulating; Reduce number of words; Reduce number of characters

46 46 Conclusions Understanding basic Gestalt principles helps manage focus Some objects on a display are more conspicuous than others  Use highlighting sparingly Some objects form ‘perceptual groups’  Use this to help design screen layout and to minimise risk of confusion

47 47 3. Link and Layout Analysis Link Analysis  Sequence in which objects are used, or attended to, during a task  Data recorded from eye-movement or from observation of product use Layout Analysis  Define relationship between objects in terms of functional grouping and sequence of use

48 48 3. Link and Layout Analysis

49 49 Eye-Movement ‘Heat Maps’

50 50 Reading Saccades and Fixations Anticipation and Inferences Interpretation

51 51 Saccades and Fixations

52 52 Reading from Computer Screens Reading Speed  Computer 20% - 30% slower than paper BUT only when  Refresh rate < 60Hz  Resolution < 640x480 pixels  Differences essentially related to character discrimination data-driven perception (see slide 27)

53 53 Link Analysis Worked Example: scan of initial layout Cassette door Radio display BAND ASPM ME-SCAN ST DX/LO A SCAN SEEK CD 123 456 B

54 54 Link Analysis Worked Example: revised layout Cassette door Radio display BAND ASPM ME-SCAN ST DX/LO A SCAN SEEK CD 123 456 B

55 55 Layout Analysis Worked Example: initial layout Cassette door Radio display BAND ASPM ME-SCAN ST DX/LO A SCAN SEEK CD 123 456 B

56 56 Layout Analysis Worked Example: functional grouping Cassette door Radio display BAND ASPM ME-SCAN ST DX/LO A SCAN SEEK CD 123 456 B RADIO CASSETTE

57 57 Layout Analysis Worked Example: redesign within functional groupings Radio display BAND ASPM ME-SCAN ST DX/LO A SCAN SEEK CD Cassette door 123 456 B

58 58 Context When presented with ambiguous stimuli our prior knowledge of the world helps us to make sense of it

59 59 Context

60 60 Interpretation Knowledge of what you are “looking at” can aid in interpretation JACKAN DJI LLW ENTU PTH EHILLTOFE TCHAPAILO FWATER Organisation of information is also useful ABC 1201510 2321612 3271711 Gestalt also influences interpretation

61 61 Concepts How do you know a chair is a chair? A chair has four legs…does it? A chair has a seat…does it?

62 62 Mental models Craik  Internal representation of external reality  Every good monitor needs a model of the world it is monitoring Johnson-Laird  Strategies of knowledge assimilation

63 63 Mental Models Van Dijk and Kintsch (1983)  Text processed to extract propositions, which are held in working memory;  When sufficient propositions in WM, then linking performed;  Relevance of propositions to linking proportional to recall;  Linking reveals ‘gist’

64 64 Inferences Comprehension typically requires our active involvement in order to supply information which is not explicit in the text 1. Mary heard the ice-cream van coming 2. She remembered her pocket money 3. She rushed into the house.

65 65 Inference and Recall Thorndyke (1976): recall of sentences from ‘Mary’ story  85% correct sentence  58% correct inference – sentence not presented  6% incorrect inference

66 66 Scripts, Schema and Frames Schema = chunks of knowledge  Slots for information: fixed, default, optional Scripts = action sequences  Generalised event schema (Nelson, 1986) Frames = knowledge about the properties of things DESIGN CAN HELP CALL APPROPRIATE CONCEPTS TO MIND, e.g., through the use of metaphor

67 67 VisiCalc Xerox Star

68 68 WIMP Windows Icons Menus Pointing devices

69 69 Metaphor - Pros WOZNY (1989) Metaphors allow users to compare knowledge of the world with what is happening in the system, and use analogy in initial reasoning with the system MARCUS (1998) “A familiar concept in an unfamiliar environment can add appeal and when users are new to a topic or application domain, seeing familiar references can reduce tension, stress, boredom, confusion, anxiety, and alienation, while increasing their self-assurance, calm, interest, engagement and dependence upon the user interface.”

70 70 Designing with Metaphors MARCUS, 1994MADSEN,1994CARROL ET AL. 1988 ORGANISE: provide simple, clear, consistent metaphors Listen to how users understand their computer Identify candidate metaphors ECONOMISE: maximise effectiveness of minimal set of metaphors Build on existing metaphors Detail metaphor / software matches with respect to user scenarios COMMUNICATE: match metaphors to user capabilities Use predecessor artefacts as metaphors Identify likely mismatches and their implications MANAGE MISMATCHESNote metaphors already implicit in problem description Identify design strategies to help minimise mismatches

71 71 Smilowitz, “Do metaphors make web browsers easier to use?”

72 72 Integral metaphor(library) ‘Standard browser’ interface (no metaphor)

73 73 Integral metaphor(travel) Composite metaphor (library + travel)

74 74 Metaphor - Cons “Those who think that one should use metaphors in design are destined to produce crappy designs.” (DON NORMAN) PIRHONEN (2003) A metaphor cannot cover the whole domain of the referent  And so limits functionality of the design (HARRISON ET AL. 1998) Once a person has learned the application, the metaphor becomes redundant HUDSON (2002) Metaphor can lead to inappropriate assumptions on error recovery COOPER (1995) Searching for the ‘magic metaphor’ can be one of the biggest mistakes an interface designer can make A Metaphor could offer a tiny boost to learnability at first but eventually limits functionality There are not enough metaphors to go around Metaphors do not scale well

75 75 4. Use of Colour and Highlighting

76 76 Contrast Effects This text is quite easy to read because the contrast between figure and ground is high This text is more difficult to read because the contrast between figure and ground is lower This text is very hard to read because the contrast between figure and ground is minimal

77 77 Galitz, W.O., 1996, The Essential Guide to User Interface Design Potential problems:  screen backgrounds being more attention grabbing than screen data  overuse of colour as a code (the colour itself meaning something to the screen viewer), which forces the user to interpret a colour’s meaning before the message it is communicating can be reacted to.

78 78 Galitz, W.O., 1996, The Essential Guide to User Interface Design Use colour to assist in formatting a screen:  relate or tie elements into groupings  break apart separate groupings of information  associate information that is widely separated on the screen  highlight or call attention to important information by setting it off from other information

79 79 Tufte, E., 1989, Visual Design of the User Interface 1. The border of an active window should be light in value (to avoid clutter with other windows), yet deeply saturated (to provide a conspicuous signal) 2. Yellow is the only colour jointly satisfying both these conditions, and therefore proves valuable for bordering windows 3. A good way to avoid colorjunk is to use colours found in nature, particularly towards the lighter side, such as blues, greys, yellows 4. Nature’s colour are familiar and have a widely accepted harmony

80 80 Apple Computer Inc., 1992, Macintosh Human Interface Guidelines Use of Colour in Windows:  Distinguishes the active window from other windows and enhances the appearance of user controls on the window frame.  Scroll bars and title bars are gray  User controls are coloured to make them more apparent  Inactive window borders are gray to make them recede into background and active windows black border stands out.  User can change colours from colour control panel  User chosen highlight colours will be used in windows and dialog boxes. Limit the number of Colours  To maintain consistency with operating system, use as few colours as possible.  Fewer colours results in less visual clutter on the screen Colours on Gray  Colours look best against a background of neutral gray  Colours will stand out more if the background and surrounding areas are gray black, or white Beware of Blue  Light Blue is the colour most difficult to distinguish. Avoided for text, thin lines, and small shapes.  If you want things to be unobtrusive, thought, light blue is the perfect colour.

81 81 Factors that Affect Contrast and Visibility VariableEffectExample  Contrast  Visibility Black print on grey  Illumination  Contrast Sensitivity (CS) Reading map in poor light PolarityBlack on white better than white on black Designing slides Spatial frequencyOptimum CS at 3 C/DIdeal size of text font given viewing distance Visual accommodationCSMap reading during night driving Motion  CS Reading a road sign while moving Wickens et al., 1998, An Introduction to Human Factors Engineering

82 82 Data-driven perception Activation of neural structures of sensory system by pattern of stimulation from environment

83 83 Theory-driven perception Perception driven by memories and expectations about incoming information.

84 84 Exercise x i. X on outside front corner ii. X on inside back corner

85 85 Visual Illusions Old Woman or Young girl? Rabbit or duck?

86 86 KEYPOINT Perception limits are set by sensory / neural mechanisms; but beyond these limits, perception can be cognitively controlled; Sensory experiences interpreted in a CONTEXT and derive from a variety of sources

87 87 Conclusions Humans constantly seek patterns in the world around them… …and use these patterns to impose meaning on what they see or hear. Good design encourages this search for pattern… …bad design makes people see patterns when none are intended.

88 88 5. Commands and Menus

89 89 Commands and Icons GREP –V ^$filea>fileb A B

90 90 Command Languages Advantages Fast Efficient Precise Concise Flexible User initiated Disadvantages Training Regular use High memory load Poor error handling

91 91 Commands Languages Command languages rely on recall  Draper (1985) UNIX users poor recall on written test, but excellent recall when using computer  Black and Moran (1982) Learn 8 commands. Performance best for ‘infrequent, discriminating’ words, e.g., insert, delete. Random words also good.

92 92 Knowledge in the Head Knowledge in the World “Menus relieve users of the need to remember command names, but not of the need to know what functions can be performed by some commands…” [Mayes et al., 1988] Knowledge held by users and recalled ‘Knowledge’ prompted by objects in world

93 93 Command Languages Guidelines Kidd (1982): Choose memorable, nonconfusable command word Use consistent command formats Keep command strings short Provide help Use natural syntax Place optional / least used commands at end of list Ensure useful defaults If errors frequent, then revert to computer initiated

94 94 Menus Advantages Minimal typing Low memory load Defined structure Disadvantages Speed Screen space Poor for data entry Computer initiated

95 95 Menus Single menu  Binary options Do you want instructions? Y / N Linear sequence  Series of interdependent menus Sequence of forms for printing Tree structure  Semantic network 8 items per menu x 2 levels for time and error

96 96 Tree structure Menu Trees  reduce error rate when compared with random  Search faster over trees than alphabetic or random  Semantic networks

97 97 Menu Guidelines Task semantics Meaningful groupings Short cuts Items brief Consistent terminology Headings relevant Limit to 8 items

98 98 Icons Concrete vs. Abstract Reference and interpretation Verbal labels can speed response

99 99 Icon design Marcus (1992)  Lexical: pixel shape, colour, blinking  Synatatics: lines, pattern, size, shape  Semantics: concrete, abstract, part  Pragmatics:legibility, utility, identification, recognition  Dynamics: highlighting, selection

100 100 Icon Guidelines Represent object in familiar manner Limit number of different icons Make icon conspicuous Ensure selection made clear Ensure icons belong to ‘families’

101 101 Working Memory Experiments

102 102 Central executive Articulatory control process Auditory word presentation Visual word presentation Phonologica l store Visual Cache Inner scribe Baddeley’s (1986) Model of Working Memory

103 103 Slave Systems Articulatory loop  Memory Activation  Rehearsal capacity Word length effect and Rehearsal speed Visual cache  Visual patterns  Complexity of pattern, number of elements etc Inner scribe  Sequences of movement  Complexity of movement

104 104 Recognition vs Recall Recall  Retrieve from memory Remembering print command Recognise  To bring back into awareness through prompt Recognising print icon Recognising print item in menu

105 105 Recognition vs Recall “The boxer chewed the meat” Recall: who chewed the meat? Recognition: did the boxer chew the meat? Recognition easier than recall More information retrieved under recognition than recall

106 106 Long Term Memory as a Semantic Network ANIMAL Has Skin Can move Eats Breathes BIRD Can fly Has Wings Has feathers FISH Has fins Can swim Has gills CANARY Is Yellow Can sing

107 107 Levels and Reaction time A canary is a canary A canary is a bird A canary is an animal A canary is a fish A canary can sing A canary can fly A canary has skin A canary has gills Collins & Quillian, 1969 0.9 1 1.1 1.2 1.3 1.4 1.5 012False Levels of Sentences Mean Reaction Time (s) Property Category

108 108 Canaries Different times to verify the statements:  A canary is a bird  A canary can fly  A canary can sing Time proportional to movement through network

109 109 Spreading Activation When searching semantic network, activate paths for search The active a node, the more easily information can be obtained from it Semantic Priming Effects

110 110 Forgetting Encoding failure  Failure of consolidation Storage failure  Disruption by new or existing information  Associative interference Two responses associated with same stimulus Retrieval failure

111 111 6. Navigation in User Interfaces

112 112 Navigating Menus

113 113 Hypertext Bush (1945) Nelson (1963) Hyperlinks between objects in document Navigation  Following links through the document

114 114 Navigation Aids

115 115 Searching the World Wide Web Scanning modesInformation needInformation seekingInformation use Undirected ViewingGeneral areas of interest; specific need to be revealed Sweeping Scan broadly a diversity of sources, taking advantage of what's easily accessible" Browsing Serendipitous discover Conditioned ViewingAble to recognize topics of interest "Discriminating" Browse in pre-selected sources on pre-specified topics of interest "Learning" Increase knowledge about topics of interest Informal SearchAble to formulate simple queries "Satisfying" Search is focused on area or topic, but a good-enough search is satisfactory "Selecting" Increase knowledge on area within narrow boundaries Formal SearchAble to specify targets in detail "Optimizing" Systematic gathering of information about an entity, following some method or procedure "Retrieving" Formal use of information for decision-, policy- making

116 116 Choice of Aid x Search Style Hammond and Allinson (1989)

117 117 Information Foraging Priolli, P.,2007, Information Foraging Theory  ‘information scent’ left on path of search by users  More users on path means more ‘scent’  Users attempt to predict what information will obtained from following a given path

118 118 6. Interaction Devices: Entering Data

119 119 Generic Forms of Interaction Select Object Drag Object Change Orientation of Object Enter Data

120 120 Conversation Kevin: Want to see that film? Brian: the uh (500 ms)… with the bloke from that other um that prison thing, [Shaw…] Kevin: [The Shawshank Redemption] Tim something – [married that bird with the eyes] Brian: [Robbins] (laughs) Yeah…. 'The Green… whatsit? Kevin: OK. There’s one at uh... (looks at watch - 1.2 s)... 20 to?

121 121 Human Communication Units of analysis  Proposition  Lack of ‘proper’ grammar Adjacency Pairs  Question-answer Turn-taking  Over-lapping Extralinguistic and Paralinguistic cues  Back channels  Gestures

122 122 Turn Taking When it is your ‘turn’? Subtle cues from partner  When they are finishing  When they want to speak Unsubtle cues from machine  Speak after beep  Respond from list

123 123 Asymmetrical Conversations Telephone  Short phrases  Task oriented  Problem of silence Expert / naïve  90% of doctors ‘turns’ are questions  Negotiation of knowledge level

124 124 Is speaking to machines natural? Turn taking  Input / Feedback  Turn taking cues Asymmetrical partners  Domain / World Knowledge  Goals  Language abilities  Interpretation of ‘tokens’

125 125 Is speaking to machines natural? Politeness  You don’t say ‘please’ to a machine Social Norms  What happens when machine violates norms?  Should synthesised speech sound human?  What are the ‘norms’?

126 126 Back channelling from machines MicroSoft Office Assistant Process Control training simulation Nodding ATM / TVM

127 127 Breakdown and Repair Redundancy Shared knowledge / anticipation Focus signals  Given/new  Deixis Comprehension signals Communication signals

128 128 Eye contact and gaze to convey interest signal comprehension and communication provide sense of engagement establish social presence

129 129 Video tunnel monitor camera mirror half- silvered mirror

130 130 7. Interaction Devices: Selecting Objects

131 131 Deixis Pointing to things using words and gestures: “Please can I have that.” “Please can I have that tart.” “Please can I have that fruit tart.” “Please can I have that strawberry tart.”

132 132 Deixis in HCI Point  Move cursor onto object  Indicate object (highlighting) Select  Click button Manipulate  Click button

133 133 Select Object Step One: Indicate Object  Enter Objects Name or I.D. Type Write Speak  Position Selector on Object Direct: touchscreen, stylus, pen Indirect: cursor (using mouse, trackball, joystick) Step Two: Confirm Selection  Press key to confirm  Press button to confirm  Tap object to confirm

134 134 Device Models Buxton’s 3-state device model State 0 State 1 State 2

135 135 Application State 0 State 1 State 2 Out of range Pen on Pen off Button up Button down select drag

136 136 Different pointing devices DeviceState0State1State2 TouchscreenX PenXXX JoystickXX MouseXX

137 137 8. Multimodal Human Computer Interaction

138 138 Divided Attention Dual tasks require people to divide attention Limited attentional resource that is shared between tasks Depends on tasks, e.g., talk and drive, sight read music and shadow speech

139 139 Allport et al. (1972) Participants presented with essay, either visual and auditory Recognition test far worse with auditory Errors in Recognition (%)

140 140 Task Similarity Interference when use same stimulus modality – visual or auditory Interference when use same stage of processing – input: central: output Interference when use same memory codes – verbal or visual Interference when use same response codes – spoken or manual

141 141 Practice and Expertise  Highly practised dual-task performance, e.g., 6 weeks read + take dictation  Expert pianists can sight-read + shadow; expert typists can touch-type + shadow Performance strategies Reduced demand Reduced resource load

142 142 Contrasting Definitions Technological  Computers can present information using different display modes and can receive information from different devices Human  Humans can receive information via different senses and perform actions using different effectors

143 143 Modalities Modality  Refers to human sensory modalities Sight Hearing Touch Smell, Taste  and response modalities Speech Manual Multimodal system  Supports HCI using more than one sensory and response modality

144 144 Types of Multimodal System Task Goal DependentIndependent ExclusiveConcurrent DependentAlternateSynergistic

145 145 Exclusive Separate tasks, each with their own goals Performance on tasks interact Example:  Surveillance system with camera control Enter data + control camera Data entry delayed by camera control

146 146 Alternate Separate tasks with shared goals Performance on tasks interact Example:  Surveillance and target identification Control camera and zoom-in on targets Identify and report targets

147 147 Concurrent Separate tasks, each with their own goals Tasks do not affect each other Example:  Target identification with vehicle control Identify targets + manoeuvre vehicle

148 148 Synergistic Separate tasks with shared goal Tasks do not affect each other Example:  Mark distance on map ‘How far from here to here?’ Pen + speech

149 149 Put That There Spoken commands Manual gestures  Bolt (1980) Put that red triangle… …there

150 150 IBM OpenSpace

151 151 QuickSet

152 152 Pointing and Speaking People do not interact multimodally with multimodal interfaces Speech and pointing is not dominant  Around 20% of conversation, and 14% of multimodal HCI

153 153 Sequencing and Timing Not all actions are simultaneous 99% of pen + speech has pen actions first

154 154 Languages of Multimodal HCI Speech: ‘place a boat dock on the east, no, the west end of Reward Lake’ Pen: [draws rectangle] ‘add dock’

155 155 Future Multimodal HCI Development of future multimodal systems “…depends on knowledge of the natural integration patterns that typify people’s use of combined input modes.” “The design of multimodal systems that blend modes synergistically depends on…” properties of different modes the information they convey how multimodal input is synchronized

156 156 9. Future Human Computer Interaction

157 157 Weiser’s Scales of Interaction Inch, foot, yard Tab, pad, board

158 158 Inch = Pad ParcTAB PDA Individual many networkable

159 159 Foot = pad Notebooks Tablet E-paper Individual several

160 160 Yard = board Smartboards Plasma displays Control mimics Public sharable

161 161 Forms of interaction Implicit interaction devices Sensors Awareness Recognition technologies Natural Explicit interaction devices Keyboard, pointing

162 162 WIMP considered ‘fatal’? Brad Rhodes  The user has screen real-estate to burn too much information on screen  The user has Fine motor control Eye-hand coordination  Digital information is a primary task

163 163 Wearable WIMP Clear mapping between user action and goal Windows  Multiple views for multiple functions Icons  Presentation of graphical signs and symbols for prompts and commands Menus  Presentation of restricted option set Pointing devices  Object selection and manipulation

164 164 Windows on Wearables Unclutter world  Minimal information views  Design display to augment world Emergent features  Design display to support pattern recognition Integrate display  Design display to interact with world

165 165 WIMP considered ‘fatal’? Brad Rhodes  The user has screen real-estate to burn too much information on screen  The user has Fine motor control Eye-hand coordination  Digital information is a primary task

166 166 Wearable WIMP Clear mapping between user action and goal Windows  Multiple views for multiple functions Icons  Presentation of graphical signs and symbols for prompts and commands Menus  Presentation of restricted option set Pointing devices  Object selection and manipulation

167 167 Windows on Wearables Unclutter world  Minimal information views  Design display to augment world Emergent features  Design display to support pattern recognition Integrate display  Design display to interact with world

168 168 Icons on Wearables Reduce text Reduce information load

169 169 Text vs. Graphics Baber et al., 1999, Mobile Networks and Applications 4 Mean Reaction Times GraphicalTextual HMD 1.45s 1.85s VDU 1.0s 1.5s

170 170 Menus on Wearables Clarify options Restrict activity Access and enable through world activity

171 171 Emergent Properties Baber et al., 1999, Interact’99

172 172 Limited Options Bristow, WECAPC

173 173 Integrate with world Baber et al., 2000, Interact’00

174 174 Pointing on Wearables Selection on display Selection in the world

175 175 Selection in the World Baber et al., 2000, Interact’00

176 176 Selection from Display Carnegie-Mellon  Rotary dial selector Accelerometer for simple gestures Speech recognition Bass et al., 1997, CHI’97

177 177 Augmenting the World

178 178 Digital Desk Real paper on desk is scanned by a camera, computer images are projected onto the paper

179 179 Graspable Objects BUILD-IT Move blocks on table Change layout of room on displays Fjeld 1999

180 180 GeoSPACE Hiroshi Ishii – MIT Physical objects to manipulate map display

181 181 Rekimoto’s InfoBoard Pick and place objects using pen Move objects between laptops or virtual storage

182 182 Data Storage Data containers from Philips

183 183 Data tiles Magnetic tiles linked to ‘data’ Projection labels tiles

184 184 Rekimoto’s Data Tiles

185 185 Haptic VR Provide touch to virtual objects using force- feedback devices

186 186 Haptic Augmentation Use real objects to control virtual representations

187 187 Handheld AR PDA used to provide ‘views’ on world Views change with PDA movement

188 188 Information Appliances

189 189 Domestic Appliances Microprocessors in cars, dishwashers, washing machines, televisions etc. Domestic appliances as ‘embedded systems’

190 190 Norman (1999) Information Appliance: “An appliance specializing in information: knowledge, facts, graphics, images, video, or sound. An information appliance is designed to perform a specific activity, such as music, photography, or writing. A distinguishing feature of information appliances is the ability to share information among themselves.” D.A. Norman, 1999, The Invisible Computer, MIT Press, p. 53

191 191 Pervasive / Ubiquitous Invisible Computer Embedded Systems Networks

192 192 Information Appliances Digital camera:  Capture image  Immediate (shared) viewing of image  Download image onto computer  Edit image  Email image

193 193 Three Axioms Design Axiom 1: Simplicity The complexity of the appliance is that of the task not the tool. The technology is invisible. Design Axiom 2: Versatility Appliances are designed to allow and encourage novel, creative interaction Design Axiom 3: Pleasurability (fun) Products should be pleasurable, fun, enjoyable. A joy to use, a joy to own. D.A. Norman, 1999, The Invisible Computer, MIT Press, p. 67

194 194 Current Trends Device marriages digital camera + telephone PDA + telephone + MP3 + digital camera Problems  Visibility Which device is currently ‘on’  Accessibility How to switch devices / modes?  Functionality What functions are actually used?

195 195 11. Distributed Cognition

196 196 Using an Abacus for Addition 6 + 2 = 8 Heaven beads = 5 each Earth beads = 1 each 8 + 2 = 10

197 197 Shopping Lists Construct list  Using the process of writing the list to support decision making Remember to consult list  Check the list during shopping  Use the list to help navigate the store Reading and interpret list  Make sure everything is bought

198 198 Shopping Lists Memory aid  Do we only buy what’s on the list?  Do we buy items not on the list? Additional tasks  Does writing the list create a new task? Modified task  Does using the list change the way we shop?

199 199 ‘Things that make us smart’ Cognitive Artefacts used to assist everyday activity  Shopping List  Knotted handkerchief  Calculator  Diary

200 200 Using Lists Construct list  Pre-computation [Hutchins] Remember to consult list  Checklists in inspection Reading and interpret list  Following checklists accurately

201 201 Cognitive Artefacts Distribute actions across time  Pre-computation Distribute actions across actors  Distributed cognition Change actions required

202 202 Artefacts & Representation Surface representation  Display and maintenance of symbols on a visible surface Internal representation  Storage and organisation of symbols External representation

203 203 Mapping Naturalness  Related to directness of mapping  Related to expertise / familiarity Appropriateness  Information should be appropriate to the task (neither more nor less)

204 204 Distributed Cognition Share knowledge and information across people, things, actions Shopping lists Ed Hutchins on calculating ships speed E. Hutchins, 1990, The technology of team navigation, In Galegher et al. Intellectual Teamwork, LEA

205 205 Calculating Ships Speed 1. D=RT, (R=D/T) using pencil and paper 2. D=RT, using calculator 3. 3-minute rule: 3-minutes = 1/20 of an hour, and 100yds in 1/20 of a nautical mile  1500 yds in 3 minutes = 15nmph

206 206 Calculating Ships Speed 4. Nautical Slide Rule

207 207 Calculating Ships Speed Knowledge-in-the-head [1]  2000 yds nautical mile  60 minutes in hour  D=RT  Transposition of equations Knowledge-in-the-head [2]  3-minute rule: schema for shortcut

208 208 Calculating Ships Speed Knowledge-in-the-World  Nautical slide rule Replace calculation with manipulation  Colleagues Draw upon experience of others Teamwork  Plotter  Bearing taker  Bearing timer-recorder

209 209 12. Collaborative Working

210 210 Traditional HCI One User One Computer One Location One Task?

211 211 Limitations of this view… Task focus on computer AND on other artefacts, e.g., paper, telephone, filing cabinet etc. Other artefacts imply more than one location Office work often collaborative SO: computer only part of the work domain and is not designed to fit with other parts

212 212 Personal Computers in the Office 1970s: development of the ‘desktop metaphor’ Objects on screen analogous to objects in world,  e.g., folders, files etc. Manipulation of objects on screen analogous to manipulation of objects in world,  e.g., open folders, put files into folders BUT: no link between virtual and real objects

213 213 Changing HCI Many Users  Communication and Collaboration  Multiple Access (WWW) Many Computing Devices  Embedded Systems  PDAs, cell-phones and beyond Many Locations  Physical location (mobility)  Virtual locations (WWW) Many Tasks

214 214 Computer Mediated Communication (CMC) Same placeDifferent place Same time (synchronous) Face-to-faceTelephone Different time (asynchronous) Post-in notesLetter

215 215 CMC Synchronous Speech: telephone, videophone, videoconference Text: SMS, chatrooms, instant messaging Asynchronous Speech: voicemail, answerphone Text: bulletin boards, newsgroups, computer conferencing Mixed with other tasks Electronic Meeting Room

216 216 Shared Representations Shared Calendars  Access appointments diary to view or edit (depending on privileges) Shared Awareness Shared Activities

217 217 Shared Awareness Portholes: Xerox 1992 Low resolution images of people in their offices Overhearing / overseeing

218 218 Shared Activities Work on same text Add comments Restructure Work on same drawing Modify Suggest alternatives

219 219 Collaborative Virtual Environments (CVEs) Distributed VR systems Individuals collaborate in virtual landscapes to share information and work together

220 220 CVEs Meeting places Multiple representations Shared context  Shared knowledge, environment, tasks, objects Awareness of others Communication

221 221 Reading Virtual Control Room Avatars Virtual Display Boards Augmented Displays Virtual Monitors To Virtual Plant

222 222 Interaction in Virtual Space With other actors With virtual objects With real objects? MagicBook project

223 223 Collaborating in real space Technology to bring the virtual world to the real world Shared representations on familiar surfaces Surfaces to support interaction

224 224 RoomWare GMD-IPSI, Wiege, Wilkhahn Source: Carroll, p. 569 ConnecTable CommChair InteracTable DynaWall

225 225 Shared Awareness Portholes: Xerox 1992 Low resolution images of people in their offices Overhearing / overseeing

226 226 Exercise #.1 1. Apple 2. North 3. Charlie 4. Nickel 5. Red 6. Banana 7. South 8. Fred 9. Cent 10. Green 11. Pear 12. East 13. George 14. Dime 15. Yellow 16. Grape 17. Wayne 18. Dave 19. Penny 20. Blue Make a note of the word numbers and enter into table How many people wrote: West or Orange? What pattern do you notice in the words?

227 227 Exercise #.2 1. Latch 2. Bream 3. Hot 4. Mayor 5. Jab 6. Busk 7. Else 8. Wage 9. Clog 10. Jowl 11. Chap 12. Big 13. Smug 14. Duck 15. Trout 16. Blot 17. Reek 18. Tape 19. List 20. Skirt Make a note of the word numbers and enter into table What pattern do you notice in the words? Did you do as well as with the first list?

228 228 Exercise #.3 1. Time 2. House 3. Mit 4. Stab 5. Solve 6. Draft 7. Say 8. Off 9. Boil 10. Court 11. Greet 12. Slot 13. Hand 14. Dirt 15. Clot 16. Stale 17. Out 18. Dumped 19. Stone 20. Dice Make a note of the word numbers and enter into table Did you do as well as with the other lists?

Download ppt "1 Interactive Systems Design MSc in Communications, Computing and Human-Centred Systems Dr. Chris Baber."

Similar presentations

Ads by Google