2. ITCS 6010 Speech Guidelines

3. 1. Errors VUIs are error-prone due to speech recognition. Humans aren’t perfect speech recognizers, therefore, machines aren’t either. Goal: Reduce the number and severity of errors.

4. 1. Errors Use Specific Error Messages Limit Background Noise Allow the User to Turn Off the Input Device Provide an Undo Capability Use Auditory Icons Use Multi-Modal Cues for Errors If Applicable Don’t Assume People Hear Everything

5. Use Specific Error Messages Bad Example System: “Say the departure date.” User: “Tomorrow.” System: “Say the departure date.” User: “I want to travel tomorrow.” System: “Say the departure date.”

6. Use Specific Error Messages Good Example System: “Say the departure date.” User: “Tomorrow.” System: “I don’t understand that date. Say the month, date and year. For example, say October 13 th, 2007.” User: “July 1 st, 2007.”

7. Use Specific Error Messages An adequate error message tells the user what is wrong, why and how to correct it.

8. Limit Background Noise Background noise is input. Computer hears the background, not the user.

9. Allow the User to Turn Off the Input Device This reduces background noise errors. For VUIs, allow the user to place the system in an ignore mode System ignores input until a keyword is spoken, i.e. “I am back”.

10. Provide an Undo Capability Build in ways for users to cancel out, go back and undo actions.

11. Use an Auditory Icon Auditory Icons are sound clips with a message. When errors occur, play an auditory icon to notify the user.

12. Use Multi-Modal Cues for Errors If Applicable Use more than one mode to signal an error, if possible. Play an auditory icon, display a message and speak a message.

13. Don’t Assume People Hear Everything Just because the system spoke it, doesn’t mean the user heard it. Say important information first or last to improve the likelihood of it being heard.

14. 2. Feedback During HCI, the user needs feedback from the computer. When a user issues a command, the system should acknowledge that the user has been heard. Users also want feedback when the system is busy.

15. 2. Feedback Supply Alternative Guesses Acknowledge the User’s Speech Show When It Is the User’s Turn to Talk Allow for Verification Use Non-Speech Audio for Transitions Use In-Progress Messages

16. Supply Alternative Guesses Users may say one word, but the computer hears a different word. (IDEAL SOLUTION) i.e. User says “Boston” and the computer hears “Austin”. The computer should respond “Did you say Austin or Boston?” This is easier said than done because you have to know all the words that sound alike in order to accomplish this for a large vocabulary.

17. Supply Alternative Guesses Repeat what the user said and allow the user to correct what was recognized. (REAL SOLUTION) i.e. User says “Boston” and the computer hears “Austin”. The computer should respond “You said Boston, is that correct?”

18. Acknowledge the User’s Speech When the user speaks, provide feedback that she was heard. Auditory Icon Go to the next option If the next option is time consuming, let the user know in advance. i.e. “I heard you, let me process your request”

19. Show When It Is the User’s Turn to Talk In a multi-modal user interface, provide the user with a visual cue that the computer is listening.

20. Show When It Is the User’s Turn to Talk In a VUI, provide the user with: Prompt Auditory Icon

21. Allow for Verification Users tend to verify more when using a speech interface versus a visual interface. VUIs should allow the user to verify what is happening and what has happened.

22. Use Non-Speech Audio for Transitions When the user issues a command that requires a transition, play an auditory icon to acknowledge the transition is under way. Avoid non-speech feedback that sounds like equipment noise.

23. Use In-Progress Messages If there is more than a 3 seconds delay between when the user issues a command and the system responds, issue an in-progress message. Repeat every 7 – 10 seconds if necessary. For best results, your in-progress messages should be informative. i.e. tell the user their position in the wait queue when it changes.

24. Use In-Progress Messages Playing a musical auditory icon in the background doesn’t work alone, but it is better than nothing. No increase in user satisfaction. Combine the verbal message with music to have the best effect.

25. 3. Confirmations Confirmations are questions you ask of the user to be sure that the user has been heard correctly.

26. 3. Confirmations Use Confirmations Appropriately Ask for Clarifying Information Use Confirmations for Destructive or Predictable Actions Be Specific

27. Use Confirmations Appropriately Don’t over confirm You could overdo the confirmations by asking for a confirmation for every input. You have to balance the cost of making an error with the extra time and annoyance in requiring the user to confirm a lot of statements.

28. Ask for Clarifying Information If the expected response has more than one known response, then you may want to clarify what the user said. i.e. “Do you want to set up an appointment or contact the person by phone”

29. Use Confirmations for Destructive or Predictable Actions If the user’s action is destructive, delete files, require a confirmation. If the user’s input prone to errors, require a confirmation. i.e. the grammar has a lot of sound alike words.

30. Be Specific If the system doesn’t recognize what was spoken, be specific about what you need. i.e. “Please repeat the date again” vs. “Please repeat”. “Do you mean December 3 rd ?” is not a good example, unless you are fairly confident.

31. 4. Social and Environmental Issues Interfaces are used in the context of people’s work and their physical environment. Therefore, you have to take into consideration where and under what conditions will your interface be used.

32. 4. Social and Environmental Issues Decide on Flexibility Consider Stress Consider Social Interaction

33. Decide on Flexibility Flexibility with respect to different tasks and users. If stressful situation, time-constraints => less flexibility If simple tasks, unpredictable => more flexibility You can also consider this as adaptability.

34. Consider Stress People react to stimuli differently when under stress. Speech recognition doesn’t work well when the user is stressed, i.e. yelling. Therefore, you have to consider the stress level of the environment. High stress => reduced navigation and amount of information

35. Consider Social Interaction You have to consider how your interface will influence social interaction between people. i.e. if your interface is being used by a salesperson while the person is speaking to a customer. Will your interface cause attention problems for either person?

36. 5. Command-and-Control VUIs that recognize a limited vocabulary of individual words and phrases spoken by the user.

37. 5. Command-and-Control Special guidelines User constraints Be brief and terse No personalization

38. User Constraints Limit the user’s input through specific prompts.

39. User Constraints Bad dialogue: System: “Welcome to the XYZ Company. We look forward to servicing your travel needs. What are the dates of travel that you would like me to check for?” User: “We are interested in traveling the first week of July, say July 1 st to July 5 th ”. The system’s statement is too open. This is a natural dialogue that humans understand.

40. User Constraints Good dialogue: System: “Welcome to the XYZ Company. Say the departure date of travel. For example, say October 1 st, 2003.” User: “July 4 th, 2003” System: “Thank you. Say the return date.”

41. Be Brief and Terse People model the length of system speech. If the system is lengthy, then the user will tend to be lengthy. The length of user speech is directly proportional to the number of recognition errors. The longer you speak, the chances of errors increases.

42. No Personalization Don’t use first person ‘I’ in prompts. If users view computer as person, leads to longer phrases. Anthropomorphism

43. 6. Continuous Speech Recognition VUIs that recognize more utterances than command-and-control system.

44. 6. Continuous Speech Recognition Use personalization Simulate a conversation Avoid modal interference

45. Use Personalization Use the first person ‘I’. Does not degrade user performance. Increases user satisfaction.

46. Simulate a Conversation Encourage turn taking as in natural conversations. When application done with prompt, pause for user’s response.

47. Avoid Modal Interference Modal interference occurs when user has to speak to different modes in same environment.

48. 7. Conversation and Prompting VUI are bound by conversation and prompting. The success of your VUI is dependent upon the conversation and prompting.

49. 7. Conversation and Prompting Choose Appropriate Words Avoid Personal Pronouns in Prompts Change Voices Appropriately Use Small Steps Avoid Long Prompts or Menus Use Progressive Prompting Make Prompt Direct and Exact

50. 7. Conversation and Prompting Allow Non-GUI Terms Allow Relative Dates Avoid Long Pauses Choose an Appropriate Speed Use Tapering Use Barge-In

51. Choose Appropriate Words Use “say” if you want the user to speak. Use “enter” if you want the user to press a key.

52. Avoid Personal Pronouns in Prompts Bad Example “Tell me your credit card number.” Good Example “Say your credit card number.”

53. Change Voices Appropriately Different voices can be used to signal a mode change. i.e. different language or different part of the application If you don’t want to change mode, use the same voice throughout. Warnings and errors should use a different voice.

54. Use Small Steps Query users in small steps, but not too small. Mixed Initiatives

55. Avoid Long Prompts or Menus This goes back to the 7 plus or minus 2 memory limitation. Don’t overload the user’s memory.

56. Use Progressive Prompting If the user’s input is not recognized, then progressively provide more specific prompts with each misrecognition. For example: System: “What would you like to do?” (BAD) User’s input is not recognized.

57. Use Progressive Prompting System: “You can check your account status, obtain market quotes or view a list of other actions.” User’s input is not recognized. System: “Say one of the following: Account Status, Quotes or List.”

58. Make Prompts Direct and Exact Tell the user exactly what you expect them to say. Get directly to the point. Avoid synonyms.

59. Make Prompts Direct and Exact Example (bad): System: “Say Activate Recording to turn on recording feature.” User: “Turn on recording.” Example (bad): System: “Say Activate Recording to activate recording feature.” User: “Activate recording.”

60. Allow Non-GUI Terms For example: delete is a GUI term … remove or erase are VUI terms.

61. Allow Relative Dates For example: next Friday, yesterday, tomorrow, next week, next month, etc.

62. Avoid Long Pauses People don’t like dead air in conversation. Use auditory icons or speech to avoid long pauses.

63. Choose an Appropriate Speed If the systems speaks fast, then the users will speak fast. Users will mimic the speed of the computer.

64. Use Tapering If you must provide the same message over and over use tapering. Reduce the prompt because the user is learning. For example: 1 st time the user logs in – System: “Say the street address, city and state of the business.”

65. Use Tapering 2 nd time the user logs in – System: “Say the street, city and state.” 3 rd time the user logs in – System: “Say the address.” Tapering is an adaptive technique that works on the basis that people will learn the input appropriate syntax to speak.

66. Use Barge-In Allow users to interrupt the computer’s speech. This is barge-in.

67. Use Barge-In When a group of users have adapted to an VUI and they barge-in, they barge-in with 2 seconds of the introduction (maybe less). So, if you have changed the options of the VUI, how do you notify the users if they barge-in within 1 second?

68. 8. Menus VUIs rely on menus as main navigation tool. Critical to usability of system.

69. 8. Menus Menus with more than two options Menus with two options Indication of complete menu Menu Order Results as Feedback Nesting

70. Menus with more than Two Options Use result-action. State result then specific action. Example: “For a list of categories, say 1. For a list of people, say 2. To access your account status, say 3. To open a new account, say 4.”

71. Menus with Two Options Use action-result. State action, then result. Example: “Press1 for yes, or 2 for no.”

72. Indication of Complete Menu Use cue to inform user that menu is complete. Example: “Make your selection now.”

73. Menu Order The order items are listed in the menu is very important. List most frequently-used options at top of hierarchy.

74. Results as Feedback Best feedback for a menu option is the results of the choice. Don’t use confirmation message. Example (bad): System: “For a list of categories, say 1.” User: “1.” System: “You chose 1. The categories are ….”

75. Results as Feedback Example (good): System: “For a list of categories, say 1.” User: “1.” System: “The categories are ….”

76. Nesting If user will need to hear all options before deciding path, use nesting (sub-menus).