1. A Study of Dialogue Management Principles Corresponding to the Driver's Workload Mind-free concept (reducing mind distraction) is important in voice telematics systems. An experiment concerning a network-distributed voice-activated telematics system was conducted. Reference model, basic principles and requirements are also discussed. Standardizations of dialogue control, voice command and effect sound are effective. A standard could be developed by complementing the dialogue principles of ISO 15005. T. Nishimoto (Univ. Tokyo), M. Shioya (Hitachi, Ltd.) J. Takahashi, H. Daigo (JARI)

2. HMI Reference Model for Telematics Systems To meet ‘ Mind Distraction ’ - mind-free ( mind-on-the-driving ) To meet ‘ Visual Distraction ’ - hands-free ( hands-on-the-wheel ), - eyes-free (eyes-on-the-road ) Background and Objective Network-Distributed Telematics Systems ( Access Open-Specced Information Services ) Autonomous or Dedicated Telematics Car Navigation Systems ( Access Closed-Specced Information Services ) - Priority Classes for Information - Prohibit to Provide Info. Not for Driving Network-Distributed Telematics System to meet Safety & Situation ( Priority, Emergency, Preference ) - Situation-Aware Control - Introduce HMI Classes

3. Select Voice Engine Depending on Request Reservation Center Web Server Weather Info. Center Web Server Restaurant Info. Center Web Server Voice-Activated Center System Voice Browser Voice Engine Internet URL / Voice Web Server Voice Engine Voice / Recognized Data Text-To-Speech Data as Reply to Request Network-Distributed Voice-Activated Telematics Service System In-Vehicle System Voice Engine Voice Engine Voice Engine Vehicle Info. Location Info. ASP: Application Service Provider ASP

4. Center System Driving & Environmental Conditions In-Vehicle System Request HMI Server - Distraction Sensing - Dialogue Control considering Info. Priority & Driver’s Skill - Load Share btw Center / In-Vehicle Driver’s Conditions Info. Tags Contents Request Contents Voice Telematics HMI System Architecture SR TTS Voice/data Driver Communications Networks Vehicle SR TTS Event-driven Multimodal Interface (SALT-like) HMI Server - Sensing - Control - Share VoIP Voice VoiceXML Documents / Interpreter VoiceXML Documents / Interpreter

5. pause dialogue pause dialogue An Example of Dialogue Control for Driver Passing Constant Cruise Right Turn Left Turn Mixing : Driving Load + Telematics Load : Driving Load TIME Load Driving Ability Remained for A Driver A’s Skill Driver B’s Skill Driving Ability Remained for B Telematics Load

6. Resume Dialogue at Winker OFF Pause Dialogue at Winker ON May Resume Dialogue by Command when Signal is Red and Vehicle Speed = 0 Dialogue Control in Case of Right Turn Vehicle Right Turn

7. (1) Control of Dialogues: - Most drivers agree that it feels much safe due to the control of dialogues. - Standardization of dialogues is also requested. (2) Pause of Dialogues: - Most drivers agree the necessity of the standardization of interruption and resumption method of dialogues. (3) Voice Commands: - Most drivers agree the necessity of preparing standard commands. (4) Effect Sounds and Announcements: - Most drivers agree the necessity and the effectiveness of the effect sounds and the announcements. - The standardization is also requested. Evaluations of the Test

8. - Dialogues between a driver and telematics service systems while a vehicle is in motion. - Use mainly voices as inputs by the driver, recognized mainly by voice recognition techniques. - Use mainly voices or sounds as outputs by the systems. - Treat services not only traffic information, POI information, news, and weather, but also various services, popular in the Internet world ( e.g. e-mail, electronic commerce, information retrieval, and entertainment ). - The reference model aims to present basic principles, requirements and recommendations, for accessing the telematics services without mind distraction. View Points of the Reference Model Scope

9. - In the dialogues for the telematics services, the important basic principle is that the skill for processing the driving load is always secured by the driver. - That is, the driver should keep his 'mind-on-the-driving' until the driving load processing is completed, namely, should let his 'mind-free' from others but the driving. - The dialogues for the telematics services should be executed within the driver’s remained skill besides for the driving load processing. Basic Principle of the Reference Model

10. - The dialogues for the telematics services are to be managed or controlled so that the skill for processing the driving load is secured first. - Then, within the driver’s remained skill the telematics load is processed. - The driving load and the telematics load are to be pre- estimated or measured real-time. - The driver’s skill or capability is to be pre-estimated or measured real-time. Basic Requirements of the Reference Model (1)

11. - The management or control is to be done considering driver’s condition and driving situations ( e.g. road, traffic, vehicle and travel conditions ) of that time. - The management or control is to be done considering the priorities of the services. - The telematics service systems' state is to be informed to the driver properly. - The final initiative of the dialogues is to be taken by the driver. Basic Requirements of the Reference Model (2)

12. ISO 14813-1 ITS (TICS) Fundamental Services: 32 Services in 8 Categories ISO 15005 Dialogue Management : Ergonomic Design Principles for Dialogues between Driver and ITS Systems Considering Driver’s Workload for Safe Vehicle Control 15006 In-Vehicle Auditory Presentation: Specifications for Presentation through Speech or Sounds to Meet Auditory or Mental Overload 15008 In-Vehicle Visual Presentation: Specifications for Image Quality of Displays for ITS Systems used while Vehicle is in Motion 15007 Measurement of Driver’s Visual Behaviour: Key Terms and Parameters for Analysis 16951 Priority of On-Board Messages: 2 Methods for Determining Priority of ITS and Non-ITS Messages 17287 Suitability Assessment: Procedure for Assessing whether Specific ITS Systems are Suitable for Use by Drivers while Driving Typical HMI Related ISO Standards

13. Complementing the ISO Standards

14. A citation from ISO 15005: 5.2.4.3.4 Recommendation TICS should provide timely visual information to the driver. An example of a supplemental recommendation: 5.2.4.3.4-1 Recommendation Voice-activated telematics systems will provide the driver with timely audible information. Example 1: In situations where the driving workload increases such as when turning right or left at an intersection, the driver will be advised sufficiently in advance of the maneuver so that the voice-based telematics service dialogue can be performed safely. Figure [Dialogue Control in Case of Right Turn] shows an example of dialogue control during the execution of a right turn, which is one situation involving a heavy driving workload. The dialogue is interrupted and resumed based on the On/Off status of the turn signal lever, respectively. "Interrupt" and "resume" commands are also provided so as to give precedence to the driver's judgment.

15. - Consensus for - Reference model of voice-activated telematcs systems - Workload measurement methods (Visual + Auditory) - Evaluations of proposing requirements/recommendations - Pause of dialog (driving situation & drivers' skill) - Standardization of voice commands - Standardization of effect sounds / announements Future Works Contacts - Takuya Nishimoto nishi@hil.t.u-tokyo.ac.jp http://hil.t.u-tokyo.ac.jp/~nishi/