Speech based Drug Information System for Aged and Visually Impaired Persons Géza Németh, Gábor Olaszy, Mátyás Bartalis, Géza Kiss, Csaba Zainkó, and Péter Mihajlik Department of Telecommunications and Media Informatics, BME, Budapest, Hungary { nemeth, olaszy, bartalis, kgeza, zainko, mihajlik 1. System summary Extend Medicine Line to other languages looking for partners (e.g. FP7 call) User behaviour analysis Extension with other functions (e.g. coupling to home care systems) 5. TTS for pharmaceutical texts 7. Evaluation4. ASR for drug names Reading medical terminology by TTS [3] required the following adaptations: Pronunciation sub-module for non-Hungarian words, abbreviations, chemical expressions. Original examples:N-hepa; hidroxi-propil-metil-cellulóz; 40 µg PGE; 800 mOsm/1; kallikrein inactivator unit; HMG-CoA; non-Hodgkin lymphoma Special prosody module for pharmaceutical texts with 34 new rules for pause and 12 new rules for prosody. This module handles long and complicated sentences in PILs, text parts between brackets, long enumerations etc. Translated example: If you feel side effects, as for example squeamishness of stomach, sweating, shaking, weakness, giddiness, dryness in the mouth, sleepiness, sleeplessness, costiveness, diarrhea, less appetite, nervousness, excitement, headache, sexual troubles, please ask your doctor to modify the dosage. 2. System components Key references [1] Henton, C. "Bitter Pills to Swallow. ASR and TTS have Drug Problems" Int. Journal of Speech Technology. 8, 2005., pp [2] Fegyó, T., Mihajlik, P., Szarvas, M., Tatai, P., Tatai, G., "VOXenter - Intelligent voice enabled call center for Hungarian", In Proc. Eurospeech 03, pp [3] Olaszy, G., Németh, G., Olaszi, P., Kiss, G., Zainkó, Cs., Gordos, G. "Profivox – a Hungarian TTS System for Telecommunications Applications", Int. Journal of Speech Technology, Vol 3-4. Kluwer Academic Publishers, 2000., pp Acknowledgements The project was realized together with the National Institute of Pharmacy, Budapest, Hungary. This work was supported by the Hungarian National Office for Research and Technology (GVOP project no ). 8. Sample dialogue (translated) User dials Machine: Welcome to the Drug Information System. Please tell the drug name after the beep! U: Algopyrin M: Algopyrin, If this is not the desired drug please press #. M: Three versions of this drug are available in our database: Algopyrin 1g/2ml injection (first); Algopyrin 500 mg pill (second); Algopyrin Complex pill (third). Please tell the desired number. U: Second M: Algopyrin 500 mg pill, Please pronounce one of the following chapter titles: What is the drug used for? Before use. How to use? Side effects. How to store? U: Side effects. Machine begins to read the chapter Side effects. 6. Automatic updater maintanence An automatic updater helps the operator to maintain the database to load new drugs or leaflets or delete old ones. A new entry can be automatically checked for proper recognition and pronunciation performance. 3. User controls Speed: normal/faster (selection at the beginning of the dialogue) Read previous sentence Read next sentence Repeat the actual sentence Jump to the beginning of the chapter Jump to the end of the chapter Stop/continue To try dial: (Hungarian only) ASR test by phone for 1321 drug names 3 male and 3 female speakers. Aged between % is the recognition rate after the first utterance. Complex system evaluation in 3 user goups under 25 years (15 persons) years (33 persons, including 7 visually impaired) over 60 years (12 persons) System evaluation results clearly show the difference between the young and the elderly generation: Group A found the voice of the TTS less intelligible than Group C. Elderly people (C) found the speed of synthetic speech mostly very good, but young people (A) regarded it too slow. People of Group C found user friendliness acceptable, and persons of group A had an even better opinion. Only question 4 was evaluated by all groups similarly. 9. Future plans Medicine Line (MLN) is an automatic Hungarian telephone information system operating in Hungary since December It is primarily intended for visually handicapped persons and elderly people with normal speech functions.[1] The caller tells the drug name, the chapter title etc., which is processed by a specialized ASR module. Medicine Line reads the Patient Information Leaflets (PIL) chapter by chapter. In Hungary there are about 5000 different medicine types approved by the National Institute of Pharmacy. The output is given by a TTS synthesizer specialized to read drug names, Latin words and pharmaceutical texts correctly. The MLN system ensures 24 hour access. General purpose recognizer [2] adapted to drug names with the following features: MFCC+BEQ acoustic features GMM-HMM acoustic models are trained on 20 hours of SpeechDat-like data (Hungarian only) ML estimation of GMM parameters up to 10 mixtures Speaker independent decision-tree state clustered cross- word Hungarian triphone models Hungarian grapheme to phoneme rules tailored to drug names + manual control on the output transcription 5 Kword vocabulary 0-gram language model Off-line recognition network optimization using WFST algorithms Real-time one-pass decoding of up to 32 channels