1

2 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, The two faces… People may use more than one metaphor to make sense of human- computer interaction The interface metaphor oThe dialogue system as an alternate means of interfacing with a computer The human metaphor oThe computer as a person with human- like conversational abilities

3 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, Example #1: interface metaphor Windows Vista ASR demo Courtesy Long Zheng ( Clues suggesting use of interface metaphor: oShort utterances oSpoken commands oCommand words often identical to menu/button text oSystem communicates via GUI oSystem often identical to standard GUI

4 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, Example #2: interface metaphor Stockholm Transport (public transport) Journey planner First released as web service (forms) Later released as telephony based speech service Frequent user, knows order of system prompts Clues suggesting use of interface metaphor: oStrictly controlled input: place names, times, yes/no (or lack thereof) oStrictly limited output: Slot filler requests, verifications, journey information oInput and output follows the web interface closely oUser refrains from social utterances and feedback

5 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, Example #3: human metaphor KTH experiment on responsive grounding Partial Wizard-of-Oz Naïve user Near-perfect system understanding (WoZ) Fast interaction, backchannels, fragmentary grounding Clues suggesting use of human metaphor: o greetings, socializing o user gives verbal feedback to system utterances o user expresses opinions o long user utterances o very high confidence in system's conversational abilities

6 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, Example #4: human metaphor TeliaSonera NICE project Partial Wizard-of-Oz Game task Multiple animated agents (Karin & Cloddy Hans) Clues suggesting use of human metaphor: o user treats Karin & Cloddy Hans differently o user calls out to system (Karin) o user gives verbal feedback to system utterances o long user utterances o user negotiates with system o high confidence in system's conversational abilities

7 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, Clashing metaphors Part 1 – User (U) speaking to a machine (S): S:Welcome to our customer care call centre, How I may help you? U: fault report S:Your call is now routed to an operator Part 2 – User speaking to the operator (O): O:Fault reports, my name is Emma U:Well hi, I am Robert... O:Yeah, hi U:...and I have a problem with… or well it has to be the phone line if I may say so because I have a very weak dial tone, you know […]

8 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, Clashing metaphors (cont’d) Part 3 – Post interview (O): O:I don’t know if you noticed, we’re testing a new system U:Uh-huh O:Instead of pressing keys you use your voice to say “fault report” U:Yeah O:How did you like that? U:Yeah, well… O:Compared to pressing keys and listening and… U:Well, I didn’t quite know what to say, but then I got it… O:You got it then, didn’t you U:Yeah O:Yeah yeah U:But it’s not like you’re given any alternatives

9 Two faces of spoken dialogue systems Jens Edlund, Mattias Heldner & Joakim Gustafson Dialogue on Dialogues, Interspeech ICSLP Satellite Workshop, Pittsburgh, Evaluation Different evaluation methods may be required for different metaphors Most discussions focus on tasks where the interface metaphor is good enough (or better), and where infotransfer efficiency is the goal In tasks where the human metaphor would be especially interesting, such as games, the conversation in itself may be a goal, and efficient information transfer less relevant We may be able to take cues from the entertainment industry, where evaluation of games and films is commonplace