M. Liu, T. Stanley, J. Baca and J. Picone Intelligent Electronic Systems Center for Advanced Vehicular Systems Mississippi State University URL: A Robust Architecture for Human Language Technology Systems

SECON 2006Page 1 of 14 ● Speech recognition and synthesis ● Natural language processing ● Machine translation Human Language Technology Overview of Human Language technologies

SECON 2006Page 2 of 14 Architecture for Human Language Technology Systems monolithic system distributed systems Speech recognition Information retrieval ●●● decompose into function components Development of HLT systems: ● Monolithic fashion ● Distributed system Natural language understanding

SECON 2006Page 3 of 14 Galaxy (DARPA) Communicator Hub Servers System initialization Galaxy Communicator is an open source architecture for constructing dialogue systems. Its target is to provide the next generation of intelligent conversational interfaces to distributed information. The goal is to support the creation of speech-enabled interfaces that scale gracefully across modalities, from speech-only to interfaces that include graphics, maps, pointing and gesture. How It Works

SECON 2006Page 4 of 14 Advantages VS Disadvantages Frequent deadlocks in the communication between servers Need for automated recovery from server failures Lack of a common user interface for all the applications Development time Complex inter-process communication Plug-and-play approach Automating server startup The need for robustness to error and improved debugging capabilities A common interface to allow users to select among applications.

SECON 2006Page 5 of 14 Automated server management ● Multiple Applications Though the communicator process monitor provides a good interface to start and terminate servers, it requires manual monitoring. ● Process Manager module Automatically starts and controls all server processes in the prototype system architecture.

SECON 2006Page 6 of 14 Architectural Enhancements – Process Manager Process Manager Client Side Server Side Speech Analysis Hub Signal Detector Data Recorder Speech Recognition Hub Signal DetectorData Recorder Speech Recognition

SECON 2006Page 7 of 14 Common Application Interface A Single Screen Interface Once the user selects an application, the Demo Selector loads and displays the user interface needed for the specific application. Requirement of a common interface Demo Selector module Process Manager module Demo Selector interface

SECON 2006Page 8 of 14 Architectural Enhancements – Robustness Improvements State Machine architecture Servers redesign Each state gets a particular message. Trap inter-process communication errors Handshaking A simple protocol Sends a signal Waits for an acknowledgement Communicator frame Wait_for_Audio_ Ready state Data_Transfer State End_Of_Utterance State Initialization state Audio Ready state Data_Transfer State End_Of_Utterance State Initialization state Audio_Ready_Ack state Audio_Ready Audio_Ready_ Ack Data Data_Ack End_Points End_Of_Utterance Ack Speech Analysis ClientSignal Detector

SECON 2006Page 9 of 14 Evaluation Experiment 1: comparing the results obtained by testing utterances from the extended pilot database on the original and the enhanced architecture. Three experiments were conducted to measure the quantitative improvements in the robustness of the system.

SECON 2006Page 10 of 14 Evaluation Experiment 2 Scenario sample: (Dialog system Application) Imagine you are in a big city to attend a conference. Once the conference proceedings are over for the day, you want to visit some sites of interest. You don’t have a map with you and have no idea about the layout of the city. Use the system to plan your trip. Procedure: Five users were asked to engage in 24 usage scenarios using the original and the enhanced architecture. After a 10- minute practice to get familiar with the functionality of the system, the user performed the scenarios. The entire experiment took approximately 1 hour 30 minutes. The user was asked to cease testing if there was a system failure or he/she exceeded the allotted time of 30 minutes.

SECON 2006Page 11 of 14 Evaluation * E stands for enhanced architecture; O stands for original architecture. Results:

SECON 2006Page 12 of 14 Evaluation Experiment 3

SECON 2006Page 13 of 14 Future work Further experiments should be conducted to obtain additional measures of the robust improvements due to the enhanced architecture. Adding other state of the art application to our existing HLT package. Enhance the Process Manager to create and manage server processes on different host machines

SECON 2006Page 14 of 14 Reference J. Aberdeen, B. George and S. Bayer, “Galaxy Communicator,” SourceForge.net, Open Source Technology Group, VA Software, Fremont, California, December 2005, ( K. Hacioglu and B. Pellom, “A Distributed Architecture for Robust Automatic Speech Recognition,” Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp , Hong Kong, April J. Baca, F. Zheng, H. Gao and J. Picone, “Dialog Systems for Automotive Environments,” Proceedings of the European Conference on Speech Communication and Technology (EUROSPEECH), Geneva, Switzerland, pp ‑ 1932, September Fredrik Olsson ``A requirement analysis for an open set of human language technology tasks''. In Proceedings of Workshop on Portability Issues in Human Language Technologies held in conjunction with the Third International Conference on Language Resources and Evaluation (LREC 2002), Las Palmas, Spain, June. W. Ward and B. Pellom, “The CU Communicator System,” Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop, Keystone, Colorado, USA, pp ‑ 1234, December