1. VoiceXML: A Field Evaluation By: Kristy Bradnum Supervisor: Peter Clayton Presented in partial fulfilment of the CS Honours Project

2. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Overview Objective of Research Background Aims & Motivation Methodology Tools Results Conclusions Questions

3. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Objective of Research My project in a nutshell: An evaluation of VoiceXML 2.0, using a range of platforms, looking specifically at its maturity as a technology and its status as an industry standard. Objective of Research

4. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Background Overview of Speech Technology Overview of VoiceXML History Role Background

5. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Overview of Speech Technology Conversion between spoken word and binary Output Previously: pre-recorded prompts Today: speech synthesis (TTS) Input Previously: DTMF (pressing keys on the phone) Today: speech recognition (ASR) Background >> Overview of Speech Technology

6. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 History of VoiceXML AT&T Bells PhoneWeb project Lucents Phone Markup Language Motorolas VoxML IBMs SpeechML VoiceXML version 2.0 full W3C recommendation 16 March 2004 Background >> Overview of VoiceXML

7. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Role of VoiceXML Background >> Overview of VoiceXML

8. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Motivation 2002 Mya Andersons Field Investigation of VoiceXML 1.0 New technology Unstable Unsuccessful Now VoiceXML 2.0 = W3C standard Nortel: maturity increasing, widely accepted Jackson: already mature Project Aims & Motivation

9. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Project Aims Investigate these claims Examine: maturity of VoiceXML 2.0 as a technology its status as an industry standard Project Aims & Motivation

10. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Methodology Approach Tools WebSphere OptimTalk BeVocal Café Analysis Cross-Platform Analysis Methodology

11. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Approach Iterative approach Set goal Evaluate outcomes Determine next goal ROSS prototype Relevant to Rhodes Product secondary to investigation Methodology >> Approach Inadequate

12. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Tools 3 approaches [Seth]: Buy Rent Build 3 environments [Beasley et al]: Hosted Simulated Web-based Methodology >> Tools

13. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Project Tools WebSphere Studio Application Developer with Voice Toolkit OptimTalk 0.9.1 BeVocal Café 2.5 Methodology >> Tools

14. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 WebSphere IBMs WebSphere Studio Application Developer with Voice Toolkit plug-in Buy approach Voice Toolkit also includes other features: CCXML developer NLU model maintenance Call Flow Builder Grammar developer Pronunciation Builder but Version problems Methodology >> Tools >> WebSphere

15. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 OptimTalk Simple VoiceXML platform Desktop standalone development environment Set of libraries interpret W3C SIF markup languages Tailored towards research Command line application Requirements: microphone and speakers Methodology >> Tools >> OptimTalk

16. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 BeVocal Cafe Good background Web-based development environment Hosted platform Rent approach Tools: VoiceXML Checker Vocal Scripter Methodology >> Tools >> BeVocal Café

17. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Analysis Each platform studied separately 15 examples for OptimTalk Basic + Blackjack 10 projects for BeVocal Café Millers 10 Projects to Voice-Enable Your Web Site Methodology >> Analysis

18. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Cross-Platform Analysis BeVocal Cafés projects in OptimTalk OptimTalks examples in the Café Run amendments through original platform Methodology >> Cross-Platform Analysis

19. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Results Platform Independence OptimTalk BeVocal Café Grammars Design Considerations Platform Certification Results

20. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Platform Independence Environments: steep learning curve Extensible tag set Limits platform independence Proprietary extensions Some features added – some left out Example code usually worked Results >> Platform Independence

21. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 OptimTalk Results >> Platform Independence >> OptimTalk

22. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Results >> Platform Independence >> OptimTalk OptimTalk

23. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 OptimTalk Demo Results >> Platform Independence >> OptimTalk

24. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 OptimTalk Speech recognition component erratic Built-in grammars not supported in OptimTalk type attribute of Boolean = yes / no grammar Number = ? No Phone numbers (from database) Results >> Platform Independence >> OptimTalk

25. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 BeVocal Café More mature Very confusing error messages Many proprietary extensions Results >> Platform Independence >> BeVocal Café

26. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004

27. Grammars Built-in grammars not supported in OptimTalk OptimTalks grammar not supported by BeVocal W3C passes responsibility to SIF Platforms should support ABNF of SRGS Results >> Grammars

28. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Design Considerations No long menus Pronunciation Different voices for TTS ROSS by LH Michael vs Microsoft Sam Be careful with ASR Ties in with grammar No break in in OptimTalk So lists run together Results >> Design Considerations

29. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Platform Certification VoiceXML Forums Platform Certification Program Test Suite v1.0 700 test programs To check compliancy with VoiceXML 2.0 3 platforms passed (in September) NVP VoxPilot Open Media Platform VoiceGenie Platform Results >> Platform Certification

30. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Conclusions Learnt a lot about Speech technology Language does seem fairly mature now Fewer extensions More complete as a standard Still not quite stable Conclusions

31. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Conclusions Give platforms time to catch up Still hurdles in development Especially in South Africa Possibly better for commercial enterprises Not for research But improving all the time Conclusions

32. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Future work Windows vs Linux comparison Look at W3C Speech Interface Framework as a whole VoiceXML 2.1 is on its way Conclusion >> Future Works

33. ------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004 Questions Questions ???