1. Pardon Me, Your Computer’s Showing Using speech to speed and streamline desktop computing Kimberly Patch President, Redstart Systems SpeechTek West February 22, 2007

2. Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps Making commands easy to remember Enabling combinations

3. Human-Machine Grammar Follows the way the brain works Designed for controlling a computer

4. Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps

5. Tap existing knowledge, but don’t let experience confine new methods of communication.

6. 11 steps Retrieving a Picture Keyboard and Mouse Steps Click Insert Click Picture Click From File Click Look In Click Program Files Click Redstart Systems Click Utter Command Click Demo Click Bird Pictures Left Arrow Enter

7. Think outside the silicon box

8. 11 steps Retrieving a Picture Keyboard/Mouse Style in the Real World Stand up Walk to the file cabinet Open the third drawer Go to the "B" section Find the Bird folder Open the Bird folder Find the Pictures subfolder Find the Redstart Singing picture Take it out of the folder Put in my hand Let go

9. Efficient Speech # 1Efficient Speech # 2 Under i p f Bird Pictures Folder 1 Right Enter Redstart Singing File All Copy to Word New 3 steps 2 steps Retrieving a Picture

10. Keyboard and Mouse Click Outlook Express icon Click Message Click New Message Type “David, Linda” Tab Key Type “Eric” Tab Key Type “Paris trip” Tab Key Type “David, Linda,” Enter Address Email and Add Subject, First Line 12 steps

11. Keyboard and MouseEfficient Speech Click Outlook Express icon Click Message Click New Message Type “David, Linda” Tab Key Type “Eric” Tab Key Type “Paris trip” Tab Key Type “David, Linda,” Enter Express David Linda CC Kim Paris trip 1 Tab 12 steps 3 steps Address Email and Add Subject, First Line

12. Accommodating vs. Direct

14. Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps Making commands easy to remember

15. How do you remember commands?

16. Two Fry

17. Counter Trey Right

18. Unit 26, Code 11-31, 13th and Vine

19. Delta 265, Clear to Land, Runway 3-0

20. Chair Dance Teeth Grow

21. Blues 2

22. Human-HumanHuman-Computer Two Fry Counter Trey Right Delta 265, Clear to land, Runway 3-0 Chair Dance Blues 2 Word Open Backspace Left Delete Bird Pictures Folder Window 0 by 20 Window Close No Efficiently Worded Instruction

23. Important Factors How the brain processes words

24. Important Factors How the brain processes words Memory chunking

25. Important Factors How the brain processes words Memory chunking Six degrees of separation

26. Human-Machine Grammar No synonyms

27. Human-Machine Grammar No synonyms Logical rules to minimize wording possibilities

28. Human-Machine Grammar No synonyms Logical rules to minimize wording possibilities Follows the way the brain uses language

29. UC Top 60 Command Words (plus numbers and labels) All Caps Another By Cap Check Clear Compound Copy Cut Go Graph Insert Menu Message Microphone Mouse New Nope Paste Redo Screen Seconds Short Site Speech Spell This Touch Touch 2 Tray 1-20 Under Undo Volume Win(dow) Word Words Paired words Left – Right Lefts – Rights Before – After Befores – Afters Up – Down Ups – Downs Line – Line Up Lines – Line Ups On – Off Open – Close Top – Bottom Max(imize) – Min(imize)

30. Most Common HMG Rules Match words used for a command as closely as possible with what the command does Be consistent Eliminate synonyms Follow the way people naturally adjust language to fit a situation Follow the order of events

31. Common Commands Speech On Line Copy 3 Before Window Close Screen Clear

32. Keys to Using Speech to Speed and Streamline Computer Control Minimizing steps Making commands easy to remember Enabling combinations

33. Computer Command Steps

36. Carnegie Mellon Study 74% of users prefer a structured rather than natural language approach to speech recognition Speech Graffiti Vs. Natural Language: Assessing the User Experience, Stephanie Tomko and Roni Rosenfeld, Carnegie Mellon University www.cs.cmu.edu/~usi/papers/HLT04.pdf

37. Command Language Based on Efficient Human Process Minimal steps

38. Command Language Based on Efficient Human Process Minimal steps Easy-to-remember commands

39. Command Language Based on Efficient Human Process Minimal steps Easy-to-remember commands Combinations

40. Beyond the Keyboard and Mouse Speech links Document access and navigation Favorites-like lists Virtual calculator

41. Word Help – Insert a Special Character Table of ContentsSearch Function 1 Click Table of Contents Click Working with Text Click Symbols, Characters…" Click "Insert a symbol or…" Click Close Function 1 Special Characters Click "Insert a symbol or..." Click "Insert a special… " Click Close 8 steps 7 steps

42. Beyond the Keyboard and Mouse Speech links Document access and navigation Favorites-like lists Virtual calculator

43. kim@redstartsystems.com www.redstartsystems.com Human-Machine Grammar Papers and presentations Utter Command – speech interface software that works the way your brain does