Presentation on theme: "MAJORDOME Gérard CHOLLET, Richard CROCE, Laurence LIKFORMAN,"— Presentation transcript:
1 MAJORDOME Gérard CHOLLET, Richard CROCE, Laurence LIKFORMAN, Dijana PETROVSKA-DELACRETAZ,Pascal VAILLANTENST/CNRS-LTCI 46 rue Barrault PARIS cedex 13
2 Majordome Outline What is it ? What it does for you ? Research and application topics:The SIROCCO projectThe EUREKA !2340 MAJORDOME projectVoIP, VoiceXML, Human-Computer InteractionPerspectives
3 Majordome is a distributed Personal Digital Assistant It is your digital slave. It is personal. It remembers everything that you told him.It uses resources from you mobile (wireless) device, from your home, from your office, from the Internet, from the environment, …You interact with him using voice, pen, graphics, …
4 Interactions with your Majordome Majordome recognizes your identity, your voice, your handwriting, ...His speech recognizer is adapted to your voice,His handwriting recognizer is adapted to your writing style,He can speak to you,He can display information for you,He can talk with other persons either locally or over the phone.
5 What Majordome does for you ? Answers your phone,Receives and interpret your faxes, your s, …Supplements your memory (address book, agenda, bookmarks, alarm clock, health record, bank account, documentation, …)Serves as an interface between you and the (digital) world,Searches the web, internet forums, …Controls your home, your car, your children, your parents, …
6 A framework: A L I S P A utomatic L anguage I ndependent S peech P rocessingwith applications inSpeech Coding, Synthesis, Recognition,Speaker Verification and Language Identification
8 SIROCCO Unlimited vocabulary speech recognition system French lexicon (MathLex) with 64kwords (AUF task)Feature extraction with Spro (G. Gravier)Context-dependent HMM phone modelsWord pronunciation graphUses CMU-Toolkit for Language modelingBeam search for word hypothesisRescoring of word hypothesis by A*
9 Unified Messaging System HolistiqueEDF«MAJORDOME»Unified Messaging SystemEureka Projet no 2340D. Bahu-Leyser, G. Chollet, R. Croce, K. Hallouli , J. Kharroubi, D. Kofman,L. Likforman, E. Matta-Sanchez, D. Petrovska, M. Sigelle, P. Vaillant, F. Yvon
10 Participants speech : G. Chollet, R. Croce, J. Kharroubi, D. Petrovska fax : K. Hallouli, L. Likforman, Marc Sigellelanguage : P. Vaillant, F. Yvonplatform : D. Kofman, E. Matta-Sanchez, R. Croceergonomy : D. Bahu-Leyser
11 Majordome’s Functionalities Speaker verificationDialogueRoutingUpdating the agendaAutomatic summaryVoiceFaxMAJORDOME(
12 Overview of Majordome Background tasks (server-side only): sorting and filtering messages from different sources ( , voice, fax, SMS,…);extracting relevant information for reporting to user (names of senders, subject,…).Dialogue with the user: over phone or Web.The system presents the state of the mailbox, the type of messages, their sender, subject, and may sum them up or read them on request;The users access their mailbox, addressbook, time schedule, or URIs (Web addresses).
13 Voice technology in Majordome Server side background tasks:continuous speech recognition applied to voice messages upon receptionDetection of sender’s name and subjectUser interaction:Identification of the speaker (and Verification if necessary)Speech recognition (receiving users’ commands through voice interaction)Text-to-speech synthesis (reading text summaries, s or faxes)
14 Distance Learning Service Voice Over IP PlatformVTHDRenaterUnisphereERX-7001Gbps (FO Interne)ENST-ParisRTC/RNISIntranetGKPBXGWIPVR(1GbpsCisco Catalyst 6507Salle C-234Salle PBXNetwork /11VideoServerDistance Learning ServiceNetwork /11Network /11Visio conference
16 Majordome / NetCentrex project PABX /Gateway ENST-Call Control Server-Application ServerCalling personNetCentrex #Usual #IP-VR NetCentrexRecorder MachineNo responseIs the calledperson here ?Usual user calledNetCentrex usercalledVocal
17 Majordome / NetCentrex project PABX /Gateway ENST-Call Control Server-Application ServerCalling personNetCentrex #Usual #Voice Interactive callNo responseMAJORDOMEIP-VR NetCentrexSpeaker verificationDialogueVocalRoutingUpdating the agendaAutomatic summaryUsual user calledNetCentrex usercalled
18 PerspectivesAdd Vision, Hearing and Understanding to Mobile Terminals (UMTS)Multimedia for Distance Education and Conference IndexingSemantic Web,‘Universal Networking Language’‘Smart Home’, ‘Smart Car’, ‘Smart Office’
19 PerspectivesThe application context of the Majordome project could be of interest to COST-278.The Majordome/NetCentrex platform could be made available to interested partners.HTK, ISIP and SIROCCO softwares are available as freeware. One of them will be used on the NetCentrex platform.