Presentation is loading. Please wait.

Presentation is loading. Please wait.

User-Focused VUI Design Susan L. Hura, PhD Principal, SpeechUsability SpeechTEK 2007.

Similar presentations


Presentation on theme: "User-Focused VUI Design Susan L. Hura, PhD Principal, SpeechUsability SpeechTEK 2007."— Presentation transcript:

1

2 User-Focused VUI Design Susan L. Hura, PhD Principal, SpeechUsability SpeechTEK 2007

3 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Agenda Preliminaries Speech, Language & Computers 101 The Design Work Before the Design Heuristics for VUI Design Usability Testing

4 Speech, Language & Computers 101

5 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design The Speech Chain “What’s my balance?” “She wants her balance.”

6 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Language “An over-learned behavior” Exposure even before birth Continual and immersive Unconscious rules And therefore unconscious expectations… Unless those expectations are violated A socio-cultural and linguistic phenomenon

7 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Rules of Conversation Mike: “Could you pass me the salt?” Sally: “Yes.” (does nothing) Mike’s intent: politely requesting that Sally pass him the salt Sally’s expected response: pass the salt Sally’s actual response is uncooperative, but logically appropriate

8 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Rules of Conversation In VUI design, we often make people give uncooperative responses and require that they do not advance the conversation

9 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Sound is the Medium

10 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Not Beads on a String…. b æ “balance…” But inter-connected puzzle pieces l æ

11 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Variance Everywhere Meaning “right” = “right” = “right” Pronunciation [ra I t]  [ræt]  [ro I t] Acoustics The point of all of this: speech recognition is hard!

12 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design How Do We Know He Said “Right”? Top-down knowledge Context Previous experience Real world knowledge Complex mappings from sound to meaning

13 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design How Does the ASR Engine Know He Said “Right”? It doesn’t. All the engine “knows” is statistics Effects of training: labeling & transcription [ra I t] = right, [ræt] = right, [ro I t] = right, etc. Acoustics “right” “right” “right”

14 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design What Does This Mean for VUI Design? You are better at speech recognition than any ASR engine Remember, for the computer, it’s all acoustics! You can still fool a recognizer So plan on recognition failures Grammars matter Set of words or phrases you expect to recognize

15 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Capture & Capture & Digitization Digitization SpectralRepresentation Caller speaks an utterance Segmentation Search & Match Search & Match Phonetic Network Lexical Network "n-best" list PhoneticClassification ao.92 b.22 ae.43 eh.32 aw.51 PhonemeProb. Sound Segment Acoustic Models Vocab & Grammar Phonetic Recognition Process

16 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design How Speech Recognition Works Caller responds to a prompt, e.g., “account balance” Speech is detected Through a process called endpointing The sound is captured, digitized, and pre-processed in a variety of ways Echo cancellation Background noise reduction The resulting “clean” speech signal undergoes spectral analysis (to produce a spectrogram) The spectrogram can then be divided into acoustically distinct segments

17 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design How Speech Recognition Works Acoustic segments are then compared against the set of acoustic models being used Not a simple one-to-one mapping There are different acoustic models for each language supported by the recognizer Even the “same” sound will be different in different languages Acoustic models are influenced heavily by training data Acoustic models are phonetically-based Phonemes Features Diphones, triphones

18 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design How Speech Recognition Works A number of different possible “phonetic paths” are calculated Paths are compared with items in the grammar Grammars contain words the user might say Result: N-best list: list of possible user utterances Confidence scores: statistical likelihood for each Likelihood based on the closeness of the match between the incoming signal and the stored representations

19 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design What We Do with the Results Use confidence scores to determine what the application should do next Upper confidence limit: point at which the application assumes correct recognition and proceeds to the next dialog state Correct recognition of in-grammar utterance False acceptance of an in-grammar utterance False acceptance of an out-of-grammar utterance Lower confidence limit: point at which the application assumes that no recognition is possible for this input and moves into error handling Correct rejection of out-of-grammar utterance False rejection of an in-grammar utterance

20 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design What We Do with the Results The tricky case: when the top item in the n- best list has a confidence score between the confidence intervals Historically recommended course of action is confirmation “I think you said ‘transaction history.’ Is that correct? Please say yes or no.” Not always necessary or smart

21 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Then Repeat Many Times The same process occurs every time we recognize speech At every dialog state--every time the recognizer is listening for a response following a prompt For all universal commands, whenever they’re spoken If the user has “fooled” the recognizer, the results are comical or annoying or disastrous Side speech Non-verbal mouth sounds Background noise

22 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Speech Recognition Evolution Touch-tone Replacement System: For information regarding the status of your return, press or say ‘one.’ Caller: One. Directed Dialogue System: You can select tax tips, FAQs, or you can check on your refund. Caller: Check on my refund. Natural Language System: How can I help you? Caller: I was wondering when I would receive my refund check?

23 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Evolution of the Speech Interface Spontaneous Dialogue System: How may I help you? User: I want to check for the cheapest flight from San Francisco to Atlanta on the 23 rd, returning on the 26 th. Slotted Grammars System: Please tell me your departure information. User: I need to leave San Francisco on February 23 rd around 4 PM. Directed Dialogue System: What’s your departure date? User: February 23rd Technical Difficulty User Control User-Led Dialogue User…..

24 The Design Work Before the Design

25 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Before the Design Information-gathering is vital, but often neglected in project plans A list of functions to be automated is not the sole goal of requirements gathering! Designers alone carry all this information and must rely on multiple sources and techniques to find it

26 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design The Role of VUI in Speech Projects A three-way intersection VUI Business Goals User Goals Technological Constraints

27 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Design as Therapy Goal of any design is creative synthesis from disparate goals, making best use of the technology Possible only with information from all three points of the triangle Designer must act as translator and message bearer to project sponsors

28 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Project Overview Information gathering phase Design based on information; design feeds testing Testing puts designs in front of real users and exposes issues

29 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Ideals in Information-Gathering Stakeholder interviews What is the business trying to accomplish? What is most important? Overall customer contact strategy? How will we know if we succeed? Technology plan Abilities, limitations? What user data do we have access to and when? What data do we need to collect from users and when? CTI: what data can we pass to agents? User interviews What are they trying to accomplish? What’s the context of use? Urgency? Critical? Private? Other points of contact? Overall technology comfort level

30 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Guerilla Tactics Stakeholders Initial design presentation as a method to flush out disagreements, educate sponsors about technical limitations, and prevent misunder- standings Technology Make friends with a developer or telephony manager Talk to call center staff Users Look for surrogate users Role play Investigate other customer contact: website, commercials, print ads, current IVR, etc.

31 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Design Strategy The logical next step after requirements gathering and analysis Definition of the “sound and feel” All the elements that contribute to the overall user experience The answer to the question “I have this requirements data, so what do I do with it?” First opportunity for testing

32 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Design Strategy as a VUI Style Guide Design Strategy establishes a set of standards against which the VUI design team can measure every design decision Give examples Wording Functionality

33 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Strategy as a Communication Tool Allows the VUI designer to.... Re-explain the design process Reenforce that the time and effort spent on requirements was worthwhile Show that each prompt is an instantiation of an overall strategy, so making any change can have repercussions And thus may help to curb client’s tendency to be an ‘armchair quarterback’ and modify prompts because it sounds better to them Design Strategy is often very impressive to clients They are often not expecting it Reenforces the expertise of the VUI design team

34 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design VUI Elements Defined High level elements that need to be defined are: Recognition Strategy Directed dialogue versus ‘how may I help you’ open-ended dialogues Sound and Feel Persona: personal characteristics that are conveyed by the application Style and flow of discourse, use of earcons, etc. Information Architecture Functionality and how it will be arranged, including how it is presented to users and how they navigate among functions There are many smaller decisions for each of these high-level areas

35 Heuristics for VUI Design

36 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Rules of Thumb 1. Make It Real: Be sure there is a match between the application and the real world. 2. It’s Not Star Trek: Clearly and consistently communicate system capabilities. 3. Talking & Listening: Minimize the limitations of the modality. 4. Give Me a Hint: Help users avoid escalating errors and recover from errors gracefully. 5. Warm Fuzzies: Make the caller comfortable with the technology.

37 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Principle 1: Make It Real Users bring in their experiences and terminology. They have a mental model of the domain and their interaction. Usable applications tap into this knowledge to give users a head start in understanding how to interact with them.

38 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.1 A Rose by Any Other Name… Is not as easy to recognize! Users must remember what to say. Hunting by trial and error is frustrating and time-consuming for users.

39 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.1 A Rose by Any Other Name Is the terminology easy for users to understand? Are the branded terms familiar enough to be comfortable? Does the system call things what users call them? Is terminology consistent for all customer contact?

40 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.2 Make the Common Tasks Faster Efficiency matters, sometimes. The more often users perform a task, the quicker it needs to be.

41 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.2 Make the Common Tasks Faster How often do users call the system? Within a system, which tasks get requested repeatedly by the same user? Which tasks are rarely used? Where are speed and efficiency most important?

42 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.3 Getting There from Here Dead ends are a disaster in speech applications. Remember that users are sometimes hunting, and sometimes the system makes mistakes.

43 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.3 Getting There from Here Is there a way for the user to back up to a known point in the application? Has it been clearly provided to them?

44 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.6 Where You End Up Decisions have to be made about where to go when a task is finished. Depends on the types of tasks the user is performing and their relationship.

45 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 1.6 Where You End Up What happens after the user is done with a task? Are they given the chance to repeat the same thing? Are they returned to the Main Menu? Are they taken somewhere related to the last task? Is it clear to them what’s happening and what their options are?

46 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Principle 2: It’s not Star Trek Clearly and consistently communicate system capabilities to the user. Interfaces need to guide users to speak predictable utterances and avoid the unconstrained conversational speech that we use talking to another person.

47 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 2.1 Asking the Right Questions Speech applications need to ask questions to lead users to give the right answers. We don’t always answer exactly the question that was asked, but the other person can generally understand what we intended. Speech applications aren’t that clever.

48 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 2.1 Asking the Right Questions Does the application ask targeted questions? Do the questions elicit a very limited set of likely responses? Do they lead users to provide the “right” kind of response?

49 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 2.2 Raising Their Expectations (for a while) “Natural Language” is here— sort of. Statistical language models allow speech applications to categorize free-form utterances and “understand” the user, without offering a limited menu of options. Costly to implement. Tend to be used only for part of the automated customer interaction.

50 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 2.2 Raising Their Expectations (for a while) Are users clearly told how they should respond in different portions of the application? Is it natural for them to do so?

51 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Principle 3: Talking and Listening Conversation between people works because we share a set of rules and assumptions about talking and listening. These rules are largely unconscious, but when they are not followed, the conversation that results is difficult to follow and uncomfortable.

52 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 3.1 Talk to Me There are unspoken rules of conversation that tell us when it’s OK to talk. We’ve all been conditioned to follow these rules and converse politely.

53 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 3.1 Talk to Me Are users given clear signals of when to talk? Do prompts phrased as statements contain a clear indication of when it’s the user’s turn?

54 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 3.2 Just the Facts, Ma’am Listening is a difficult task. Auditory memory is limited. The implications for readouts in a voice user interface are substantial.

55 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 3.2 Just the Facts, Ma’am Are there other demands on the user’s attention? How many items can they remember? How familiar are the terms? How long is each item? How long can they remember them?

56 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 3.3 Where Did You Go? Dead air happens in systems and is sometimes unpredictable. Our natural rules of conversation don’t leave large gaps without an indication of what’s going to happen.

57 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 3.3 Where Did You Go? How much dead air will users tolerate before anxiety levels rise? Did you tell them it’s coming? Did you make sure there’s at least a little silence after the warning?

58 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Principle 4: Give Me a Hint Automated systems have limitations. We aim to make the interaction intuitive and simple, but there are many cases in which users will require explicit instructions to use a speech application effectively. For users to make good use of instructions, they must be presented concisely and at the appropriate time in the call flow.

59 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.1 Tell Them What They Need to Know Users need (and want) enough instructions to effectively use the automated system. Telling them just enough but not too much is a delicate balance.

60 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.1 Tell Them What They Need to Know Are there enough instructions? Do users know what they can say?

61 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.2 When They Need to Know It There is no permanent list of commands or instructions in a speech system. Users can’t refer back to a list. They must remember what they can say.

62 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.2 When They Need to Know It Are users being burdened with a big list up front and expected to remember it? Are they being provided with instructions a little at a time? And are those instructions given just-in-time, when the user is about to need it?

63 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.3 Only If They Really Need It Too much is as problematic as too little. Memory, memory, memory.

64 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.3 Only If They Really Need It Are users overwhelmed with too much information? Is all the information pertinent? Do they really need to know it? Is the design trying to accommodate too many edge cases?

65 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.4 For Instance… Examples speak louder than words, in many cases. Easy to understand and remember. Can be a very effective way of providing instructions.

66 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 4.4 For Instance… Is the task easy to explain? Is explaining what to say taking longer than modeling it would?

67 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design Principle 5: Warm Fuzzies Most users of speech-enabled applications are not calling to use the technology for its own sake. They tend to care more about accomplishing their goals than about cool technology. Without appropriate reassurance, users wonder if their spoken input was understood and if their transactions will be processed appropriately.

68 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.1 Who Are You? Many techniques to identify the user. Account number and password ANI Speaker verification Some are more familiar and comfortable than others. Identifying themselves is not the users’ motivation for calling.

69 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.1 Who Are You? Do users understand how the automated system is identifying them? Do they trust that the system has them correctly identified? Is the system engaging in overkill for identification and verification?

70 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.2 “I Don’t Mind Interruptions…” In polite conversation, we’re taught not to interrupt. Touchtone IVRs allow rampant barge-in. Users bring both mental models into interactions with speech-enabled IVRs.

71 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.2 “I Don’t Mind Interruptions…” Which mental model is the user base more likely to align with? What kind of tasks are they performing? How frequently do they use the application? Based on the answers to the above, is it worthwhile to let users know that they can barge-in?

72 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.3 Enough Already! Users want to know that they were understood correctly. Too much confirmation gets in the way.

73 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.3 Enough Already! Is there a more efficient way to confirm than step by step? Is there ever a danger of confirming the same thing twice? Is it easy to correct something that wasn’t understood? Are users annoyed by the confirmation? Are they confident the right thing is happening?

74 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.4 Operator! Businesses sometimes have a goal of 100% automation, meaning users never need to speak to a live representative. Users want access to a real agent, even if their own tasks can be handled in the automated system.

75 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design 5.4 Operator! How often do users’ tasks fall outside the realm of the automated system? Where and how often should the operator be presented? How are the users who want an operator feeling about the company and the automated system?

76 Usability Testing

77 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design The Philosophy of Testing Test early and often. Functional testing is important. Usability testing is more important. An architect’s most useful tools are an eraser at the drafting board and a wrecking ball at the site. – Frank Lloyd Wright

78 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design The Philosophy of Testing The rule of thumb in many usability-aware organizations is that the cost-benefit ratio for usability is $1:$10-$100. Once a system is in development, correcting a problem costs 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design." Gilb, 1988 "The average UI has some 40 flaws. Correcting the easiest 20 of these yields an average improvement in usability of 50%. The big win, however, occurs when usability is factored in from the beginning. This can yield efficiency improvements of over 700%." Landauer, 1995

79 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design How and When to Test The earlier the better, but….. Tradeoff between testing early and how representative the data is Data collected early is suggestive, not predictive Data collected late has greater ecological validity, but necessarily involves re-work The keys to testing success: Iteration Multiple methods

80 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design What I Mean by “Usability Testing” Real users Completing representative tasks, based on realistic goals In a controlled setting Interacting with a version of system that is as close to reality as possible

81 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design The “Why” of Usability Testing Hard data User opinions based upon actual interactions Known motivation Ability to filter out past experiences Ability to ask ‘what were you thinking?’ or ‘how did you feel when that happened?’ Usability testing allows an early, low risk, cost effective preview of user reactions to an automated system We discover surprising things

82 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design What to Measure Both objective measures of user performance and subjective opinion-based measures are vital for a complete assessment of the usability of a speech application. Objective measures provide important information about how users navigate and respond to a VUI task completion rate number and location of errors number of attempts required to complete a task But this data alone only tells us when and where problems occurred, not how important they are

83 SpeechTEK 2007 Tutorial 23 August 2007User-Focused VUI Design What to Measure Opinion data is what tells us how significant the problems were from the user’s perspective. Not all issues are created equal for users, especially for new technology with which they have little experience. This points out the importance of allowing users to give some freeform opinion responses so that they can tell us what was most significant to them the importance of matching opinion data back with the experiences of a particular user.

84 Questions? susan.hura@speechusability.com 404.702.4723


Download ppt "User-Focused VUI Design Susan L. Hura, PhD Principal, SpeechUsability SpeechTEK 2007."

Similar presentations


Ads by Google