Presentation is loading. Please wait.

Presentation is loading. Please wait.

MAGA: a Mobile Archaeological Guide at Agrigento 1 DINFO - Dipartimento di ingegneria Informatica University of Palermo Viale delle Scienze - 90128 Palermo.

Similar presentations


Presentation on theme: "MAGA: a Mobile Archaeological Guide at Agrigento 1 DINFO - Dipartimento di ingegneria Informatica University of Palermo Viale delle Scienze - 90128 Palermo."— Presentation transcript:

1 MAGA: a Mobile Archaeological Guide at Agrigento 1 DINFO - Dipartimento di ingegneria Informatica University of Palermo Viale delle Scienze Palermo - Italy 2 ICAR - Istituto di CAlcolo e Reti ad alte prestazioni branch of Italian National Research Council Viale delle Scienze Palermo - Italy Agnese Augello 1, Antonella Santangelo 1, Salvatore Sorce 1, Giovanni Pilato 2, Antonio Gentile 1, Alessandro Genco 1 and Salvatore Gaglio 1,2 SpeechTek ’07 - New York City - 21 august 2007

2 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Summary Aim of the work Enabling Technologies MAGA System architecture Implementation details Examples Conclusions

3 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Aim of the work To build smart multimodal systems for information retrieval user-friendly suitable for different mobile devices We propose the use of a verbal interface (for interactions) along with RFID (for in-door positioning purposes) Requirements: Natural language interface Availability of the system in environments where mobility of the user is essential ( e.g.: the Archaeological Museum of Agrigento )

4 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Enabling technologies  Personal Digital Assistants (PDAs)  Multimodal Browsers (Verbal Interaction)  Conversational Agents (Chatbots)  Semantic Networks (Reasoning Capabilities)  Radio Frequency Identification (RFID)

5 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Personal Digital Assistant (PDA) Mobile devices and wireless communication allow people to access information everywhere Information retrieval on small devices is limited Boring interaction and large waste of time during information research!!! I/O interface

6 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Verbal interaction Experimental results demonstrate that verbal interaction allows people to reduce the time spent on manual input (Lyons et al, 2005) Vocal interaction is a more direct approach, but it comes with a set of constraints: processing power speakers adaptation narrow dictionaries bounding grammars

7 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Conversational Agents A simple approach to build a dialogue system Pattern-matching based rules Chatbots can be used as interfaces for several applications (e.g.: entertaining, educational, e-learning …)

8 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Semantic networks Allow to: describe a specific domain Concepts, properties of concepts, relations between concepts are clearly defined in that domain represent c ommonsense knowledge covering aspects of everyday life make inference make inference to generate new knowledge from the existing one share and reuse common understanding of the structure of information Es: OpenCyc, OpenMind, WordNet…

9 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 RFID Passive tags: limited reading range -> limited position estimation error Useful in environments with spot-areas

10 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 MAGA Integration of chatbot, commonsense reasoning, speech recognition/synthesis and RFID technologies to build a virtual guide accessible: in a cultural heritage environment (such as the “Archaelogical Museum of Agrigento”) by means of different mobile devices by simulation on a multimodal browser

11 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 MAGA System Architecture

12 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 RFID tag detection Tag #1 Tag #2 Tag #3 Tag #4 Database

13 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Verbal interaction engine

14 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Verbal Interaction: X+V Technology XHTML+Voice (X+V) as implementation language Multimodal Tools for WebSphere Studio V5.1.2:  IBM Multimodal Toolkit  IBM Multimodal Browser, (X+V interpreter) X+V page

15 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 X+V Client-server architecture

16 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 X+V and Grammar Creation PHP for X+V creation php request (question) ChatBot XHTML code ChatBot (question) question in textual form VXML code with the reply replay in textual form VXML code with grammar XHTML code PHP for Grammar creation MXML Rule Pre php request (domain) Rule Key-Word Rule Post Domain Database GRXML info domain

17 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Grammar example Archeological Museum Talamone Acropolis tell me i need information to when about the is dated in Agrigento

18 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Chatbot Knowledge Base

19 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Chatbot technology: ALICE Chatbots systems: dialogue based on natural language ALICE free software technology ALICE is an artificial intelligence, natural language chat robot The knowledge base is composed of question-answer modules, called categories and structured with Artificial Intelligence Mark-up Language (AIML)

20 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Each category is a rule for matching an input and converting to an output, and consists of a pattern, which represents the user input, and a template, which represents the chatbot’s answer. WHAT IS A CIRCLE> A circle is the set of points equidistant from a common point called the center The chatbots’ knowledge is stored in a tree called GraphMaster Alice’s brain Chatbot technology: ALICE

21 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 OpenCyc Ontology Knowledge base: logic assertions, common sense rules and heuristics for the reasoning about quotidian life objects and events Inference system: is based on a series of heuristic related to a reasoning techniques about microtheories CycL: the language in which Cyc (and OpenCyc) has been written, is a formal language from Lisp language

22 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 The “AgrigentoMuseumMt” created in OpenCyc

23 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Example: a Microtheory

24 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 ALICE-KB and Cyc Inference capabilities We have written a java application that bridges the AIML interpreter with the OpenCyc inference engine (based on the CyN project). The knowledge base of the chatbot is then enriched with new AIML tags that enable the ontology querying, executing and asserting CycL/SubL statements. This allows (common sense) reasoning capabilities inside Alice chatbot. A template can be seen as a meta-answer that must be processed to compose the appropriate response.

25 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 ALICE-KB and Cyc: Inference Capabilities The created ontology enables the composition of answers that are not present in the traditional AIML knowledge base. It is possible to: a) extend the number of AIML categories to obtain a more fluent dialogue; b) extend the ontology with new concepts, facts and relations of the specific domain to provide the chatbot a smarter behavior.

26 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 An example of AIML Category… I NEED INFORMATION ABOUT THE * (cyc-query '(#$Comment It is located at (cyc-query '(#$located_in ?X)) and has been created by (cyc-query '(#$createdBy ?X)) …and the resulting interaction User: I need information about the Archeological Museum in Agrigento. Chatbot: It is one of the most modern museums in Sicily and it contains archeological finds from the provinces of Agrigento and Caltanissetta, extending from prehistoric to Roman times. It is located at Agrigento, and has been created by the architect Franco Minissi.

27 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 An example of AIML Categories using the RFID information LOCALIZATION DETECTED You are in java RFID “showcase” INFO SHOWCASE INFO SHOWCASE * The showcase contains (cyc-query '(#$Contains ?X)). INFO ARTIFACT INFO ARTIFACT belongs to (cyc-query '(#$CreatedDuring ?X)), it is attributed to (cyc-query '(#$CreatedBy ?X)) and and represents (cyc-query '(#$Representing ?X))

28 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 A dialogue example User: I need information about the Archeological Museum in Agrigento. Chatbot: It is one of the most modern museums in Sicily and it contains archeological finds from neighbors of Agrigento and Caltanissetta, ranging from prehistoric to Roman times. Chatbot: You are in room #3, in front of the showcase #17. Do you want more information? User: Yes. Chatbot: In this showcase you can see an Attic amphora. It belongs to Arcaic Greek period, it is attributed to Dikaios and represents Athena. User: What is in the next room? Chatbot: In the room #4 there are Capitals and gargoyles with lion's heads from various temples.

29 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Demo

30 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Conclusions and future work Chatbot with “reasoning capabilities” and speech recognition technologies have been embedded in a Personal Digital Assistant to assist user in visiting historical sites Natural interaction overcomes the limitation of traditional pre-recorded audio/visual guides Future work will regard the enhancement of the system: Enhance the chatbot reasoning capabilities and natural language interaction; Use of different wireless technologies for context-aware service provision Use of user profiles for ad-hoc service composition

31 MAGA: A Mobile Archaeological Guide at Agrigento SpeechTek ‘07, NYC 21/8/2007 Thank you! Vocal User Interface, System Architecture Design:  Antonella Santangelo  Antonio Gentile Information retrieval, reasoning:  Agnese Augello  Giovanni Pilato  Salvatore Gaglio RFID, pervasive access, HCI:  Salvatore Sorce  Alessandro Genco


Download ppt "MAGA: a Mobile Archaeological Guide at Agrigento 1 DINFO - Dipartimento di ingegneria Informatica University of Palermo Viale delle Scienze - 90128 Palermo."

Similar presentations


Ads by Google