1. 金聲玉振 Taiwan Univ. & Academia Sinica 1 Spoken Dialogue in Information Retrieval Jia-lin Shen Oct. 22, 1998

2. 金聲玉振 Taiwan Univ. & Academia Sinica 2 Outline  Introduction  Spoken Dialogue Technologies  Spoken Dialogue in Information Retrieval  Information Retrieval in Spoken Dialogue  Some Speech Information Retrieval Examples  Conclusion

3. 金聲玉振 Taiwan Univ. & Academia Sinica 3 Introduction Speech Processing Applications Command control Data entry Consumer electronics TelephonySynthesis Verification/ Identification

4. 金聲玉振 Taiwan Univ. & Academia Sinica 4 Introduction  Using speech in information retrieval system Information Retrieval System Database Retrieved documents Input queries Text input / Speech input Text output / Speech output Text database / Speech database Platform : Computer Consumer electronic products Telephone

5. 金聲玉振 Taiwan Univ. & Academia Sinica 5 Introduction  The impact of speech in information retrieval system –Speech input : data entry –Speech output :synthesis –Multimedia database : including speech, text, image –Platform : »Consumer electronic products : PDA, remote controller, GPS... »Telephone –Personalization : speaker identification/verification

6. 金聲玉振 Taiwan Univ. & Academia Sinica 6 Introduction  Advantages using speech –Naturalness –Penetrability –Convenience –Platform flexibility  Drawbacks using speech –Probability –Variability –Security

7. 金聲玉振 Taiwan Univ. & Academia Sinica 7 Spoken Dialogue  Spoken Dialogue system –Telephony-based speech interactive system Telephony-based Speech Recognition Acoustic models Lexicon Language models Language Understanding Syntactic models Semantic models Concept grammar Dialogue Control Database Discourse context Language Generation Speech Synthesis

8. 金聲玉振 Taiwan Univ. & Academia Sinica 8 Spoken Dialogue  Related Research Topics in spoken dialogue –Telephony based speech recognition »Robustness in dealing with both speaker variability and environment variability –Language understanding »Corpus-based statistical approach »Rule based syntactic/semantic driven approach –Dialogue control –Language generation / speech synthesis –Spoken dialogue system evaluation –Architecture development in spoken dialogue system »Client-server platform

9. 金聲玉振 Taiwan Univ. & Academia Sinica 9 Spoken Dialogue in Information Retrieval  Data entry platform –Wired/wireless telephone  Output retrieved documents using speech –Precision rates are much more important than recall rates  Dialogue control technology helps Interactive communication between human and machine  Language understanding technology improves query input from text to concept  Speaker verification/identification using speech can help security for personalization

10. 金聲玉振 Taiwan Univ. & Academia Sinica 10  Input queries are usually proper nouns, which are out-of-vocabulary terms in the lexicon for speech recognition  Non-grammatical and very short input queries leads to the degradation in language modeling  Speaker independence over the telephone is highly desired  Confidence measure (probabilistic / deterministic) due to speech recognition errors  Speech coding/transmission in internet Information Retrieval in Spoken Dialogue

11. 金聲玉振 Taiwan Univ. & Academia Sinica 11 Some Examples  Air travel information service (ATIS) in USA –Best performance - at the end of 1995 »Understanding error rate of 5.9% and 8.9% for text and speech input, respectively  Flight and train schedule inquiry system in Europe –Multilingual : English, French, German, Italian  MIT Galaxy weather inquiry system  Telephone Directory Services for Financing /Banking Organizations in Taipei Area

12. 金聲玉振 Taiwan Univ. & Academia Sinica 12 Conclusion  What role can SPEECH play in information retrieval ?  What impact can spoken dialogue make in information retrieval ?