1. Ivan Elhart - ECE 992 Ubiquitous Computing University of New Hampshire 10/09/2008

2.  Mobile settings often contain non-stationary noise which cannot be easily canceled  Speakers tend to adopt to surrounding noise in acoustically unhelpful ways  New multimodal interface that will help voice search applications to recover from speech recognition errors  A multimodal interface for mobile voice search that incorporates speech with touch and text may increase recovery rates of the search

3.  Yu, D., Ju, Y.C., Wang, Y.Y., Zweig, G., & Acero, A. 2007. Automated directory assistance system: From theory to practice. Proc. of Interspeech.  Ainsworth, W.A. & Pratt, S.R. 1992. Feedback strategies for error correction in speech recognition systems. International Journal of Man- Machine Studies, 26(6), 833-842.  Hsu, P., Mahajan, M. & Acero, A. 2005. Multimodal text entry on mobile devices. Proc. of ASRU.

4.  N –best list, whenever recognition is less than perfect  Coupling of speech with touch and text  Leveraging of any partial knowledge

5. 1. Word palette Allows users to select any word of a recognized phrase

6. 2. Text hints Resort to speech whenever search entry is too long or when enough text hints have been provided

7. 3. Verbal wildcards Partial knowledge queries

8.  Search Vox architecture

9.  Simulation experiments Utterances collected from Microsoft Live Search Mobile Automated directory assistance, maps, driving directions, movie times, local gas prices

10.  Word palette

11.  Text hints

13. Ivan Elhart - ECE 992 Ubiquitous Computing University of New Hampshire 10/07/2008

14.  How does the outdoor public tangible interface support simultaneous participation and interaction of multiple users?  To provide first insights into how users approach, participate, and interact on a large multi-touch display in a public space  Observational studies in urban environments could help in understanding how multi-touch screens can affect and support social interactions  Public interactive multi-touch displays can potentially restructure the way people experience and use the space around them

15.  Semi-public displays

16.  Public displays

17. City Wall Installation in Helsinki Direct manipulation Non-modality

18.  Data collection  A continuous interaction log was written  Interactions were recorded with a web camera in a 640x480 resolution  Twelve on-site interviews were conducted  Data analysis  Combination of the video and interaction log  First time and returning users  Sessions (a ten second gap between interaction) Duration Number of active users Number of passive bystanders

19.  Findings on how the City Wall was used and how the users interacted with each other at the screen  8 days of interaction, 8.8% the display was used, 1199 users, 516 sessions, and 202 passive bystanders 72%18%23%

20.  Noticing the Display (when the wall was used)  Multi-user interactions (fun to use with others and friends) Rain shelter Stepwise app. Parallel useTeam work

21.  Conflict management  Social configurations (or roles) Withdrawal Social inter. ComedianTeacherLeaving a mark

22. City Wall Noticing Installation Parallel useConflicts