13 th Japan-Germany Forum on Information Technology German Research Center for Artificial Intelligence, DFKI GmbH Stuhlsatzenhausweg Saarbruecken, Germany phone: ( ) /4162 fax: ( ) WWW: Wolfgang Wahlster Virtual Sales Agent for Personalized Internet Shopping

© W. Wahlster, DFKI First GenerationSecond GenerationThird Generation Static Web Sites Fossils cast in HTML Interactive Web Sites JavaScripts and Applets Database Access and Template-based Generation Dynamic Web Sites Virtual Webpages Netbots, Information Extraction, Presentation Planners Adaptive Web Sites User Modeling, Machine Learning, Online Layout Three Generations of Web Sites

© W. Wahlster, DFKI respond immediately to interruptions criticism and clarification questions direct manipulation INTERFACE AGENTS RE- ACTIVE appear as life-like characters plan interactive behavior autonomously can initiate interaction PROACTIVE anticipate the user's needs adopt the user's goals provide unsolicited comments INTER- ACTIVE understand the user‘s requests answer clarification questions allow mixed initiative dialogs What are Virtual Sales Agents?

© W. Wahlster, DFKI Softbots Indices, Directories, Search Engines WWW Traveller’s Netbot: Tries to achieve traveller’s goals (finding and executing plans) checks availability finds best price uses personal preferences (e.g. frequent flyer programme, seating preferences) lets the traveller know, when seats become available (active help) Mass Services Personal Assistants e.g. MetaCrawler The Idea of Personalized Netbots

© W. Wahlster, DFKI Consumer Provider sells Information Goods Services buys Information Goods Services Web Sites Knowledge about: Usage Patterns User Models Consumer Profiles Netbot Intelligent Parallel Retrieval Information Extraction and Summarization Personalized Presentation Matchmaking Teleshopping Assistance Telemarketing Assistance Translation Services Data Mining Services Intelligent Web Services

© W. Wahlster, DFKI Advanced WebCommerce Advanced WebCommerce Virtual Web Pages One-to-One Marketing One-to-One Marketing Intuitive, Multilingual Access Dialogue with Virtual Sales Agents Shopbots for Automated Comparison Shopping Text Analysis and Generation User Modeling and Language Generation Coordinated Text & Graphics Planning Robust Dialogue Understanding Advanced Speech Synthesis Information Extraction from HTML/XML Documents Machine Translation Multimodal Interfaces Multimedia Presentation Planning Intelligent Agent Technology is a Prerequisite for Advanced WebCommerce

© W. Wahlster, DFKI Virtual Market Places with Human and Machine Agents

© W. Wahlster, DFKI 1 Research Net WWW Mobile Internet Services Embedded Internet Agents Five Generations of Internet Applications Internet Access via WAP and UMTS devices 2000 t Every Car has a homepage, Agents are main Internet users, Ubiquitious Computing

© W. Wahlster, DFKI Towards Mobile and Speech-based E-Commerce Using WAP Phones WAP phones (Wireless Application Protocol for Cellular Phones) WML as a markup language for interactive content Mobile access to virtual shops allows price comparisons during real shopping Multimodal dialog: Voice In (Speech) - Web Out (Graphics, Hypertext) Voice input using advanced speech understanding technology Easy to use: customers simply say what they want

© W. Wahlster, DFKI A Virtual Web Page is generated on the fly as a combination of various media objects from multiple web sites or as a transformation of a real web page. looks like a real web page, but is not persistently stored. integrates generated and retrieved material in a coordinated way. can be tailored to a particular user profile and adapted to a particular interaction context. has an underlying representation of the presentation context so that an Interface Agent can comment, point to and explain its components. Virtual Memory, Virtual Relation, Virtual Reality... What is a Virtual Web Page?

© W. Wahlster, DFKI Virtual Webpage Retrieved from 5 Different Servers

© W. Wahlster, DFKI Yahoo Weather Server PAN Travel Agent Andi Car Route Planner Yahoo News Server Gault Millau Restaurant Guide Hotel Guide AiA: Information Integration for Virtual Webpages

© W. Wahlster, DFKI Netbot PAN Trip Data Pictures and Graphics Pieces of Text Coordinates for Pointing Gestures Input for Speech Synthesis Icons for Hyperlinks Hotel Agent Map Agent Address Weather Agent Train & Flight Scheduling Agent Major Event Agent Virtual Web Presentation Constraint- based Online Layout Presentation Planner Persona Server Components of virtual Webpages AiA The Generation of Virtual Webpages with PAN and AiA

© W. Wahlster, DFKI Persona as a Personal Travel Consultant

© W. Wahlster, DFKI A Natural Language Agent for Finding Pre-Owned Porsche Cars Boxter, not red, must have AC, less than 20k

© W. Wahlster, DFKI System is able to flexibly tailor product presentations to the individual user and the current situation. Enhanced ECommerce through Personalization An animated character serves as “Alter Ego” of the presentation system. Personalized Presenters at DFKI

© W. Wahlster, DFKI Persona Server Behaviors Presentation Gestures Reactive Behaviors Idle-time actions Navigation actions Auditory Characteristics Sound effects, auditory icons Voice: male, female Visual Appearances Hand-drawn Cartoon Bitmaps Generated Bitmaps from 3D-Models Video Bitmaps PPP’s Persona Server implements a generic Presentation Agent that can be easily adapted to various applications

© W. Wahlster, DFKI Classification of Persona Gestures Talking Posture 1 cautious, hesitant appeal for compliance avoids body-gestures Talking Posture 2 active, attentive self-confident uses body-gestures Gesture Catalogue

© W. Wahlster, DFKI take-position (t 1 t 2 )point-to (t 3 t 4 ) move-to (t 1 t 2 )r-stick-pointing (t 3 t 4 ) High-Level Persona Actions Context-Sensitive Expansion (including Navigation Actions) Decomposition into Uninterruptable Basic Postures r-turn (t 1 t 21 ) r-step (t 21 t 22 ) f-turn (t 22 t 2 ) r-hand-lift (t 3 t 31 ) r-stick-expose (t 31 t 4 ) Bitmaps... Context-Sensitive Decomposition of Persona Actions

© W. Wahlster, DFKI Production Act Presentation Act Introduce Create- Graphics S-Show S-Wait S-PositionElaborate-Parts S-Create- Window S-Depict Label S-PointS-Speak S-Point Qualitative constraints:Create-Graphics meets S-Show,... Metric constraints:1 <= Duration S-Wait <= 1,... Distinction between production and presentation acts (i.e. Persona- or display acts) Explicit representation of qualitative and quantitative constraints Extensions of the Representation Formalism

© W. Wahlster, DFKI Objective: Enable non-professional computer users to populate their web pages with lifelike characters PET comes with: a set of characters and basic gestures an easy-to-learn Persona markup language Developer’s PET will include: a character design tool which enables users to build their own characters Technical Realization: Based on XML and Java PET: Persona-Enabling Toolkit

© W. Wahlster, DFKI Specification of the character to be used Specification of Persona actions Persona Test Features: –XML-based –easy to learn The Persona Markup Language

© W. Wahlster, DFKI URL of Webpage with Persona Tag Persona Engine Behavior Monitor Character Composer Event Handler Persona Test Persona Scripts waitscreen 4 gesture greet 0 0 null gesture laugh 0 0 null... Audio Data Bitmaps PET Application Server PET Parser PET Generator Webpage with Reference to Java Applet Functional View of PET

© W. Wahlster, DFKI Text Input Speech Input Menu Input Direct Manipulation Input Web Persona Triggers actions of the Persona Triggers operations on elements of the webpage Mouse Clicks Mouse Movements The Bidirectional Control Flow on Persona-Enabled Webpages

© W. Wahlster, DFKI Porsche 9 11 & Boxter

© W. Wahlster, DFKI some HTML elements  Active Images An active image starts a persona action when clicked.  Addressable Objects An addressable object is an object which can be addressed and manipulated by Persona via its name and its position. Persona Active Elements (PAE)

© W. Wahlster, DFKI A Virtual Sales Agent for OTTO – World’s Largest Tele-Ordering Company

© W. Wahlster, DFKI WML- Browser MS-Agent Controller WML SMIL Agent Script PET- PML PET Persona Player SMIL Player Presentation Planner DFKI’s Ecommerce Presentation Planner has been extended to accommodate for various target platforms through the introduction of a mark-up language layer

© W. Wahlster, DFKI Information Extraction Agents Information Filtering Information Retrieval Information Integration identify relevant documents wrappers –... – identify and extract relevant pieces of information – transform them into canonical form wrappers operational descriptions of a target concept abstract from concrete occurrence within document robust against modifications wrappers operational descriptions of a target concept abstract from concrete occurrence within document robust against modifications

© W. Wahlster, DFKI Use of a Life-like Character for Electronic Commerce Digital Assistant Selector

© W. Wahlster, DFKI Simulated Dialogues as a Novel Presentation Technique Presentation teams convey certain rhetorical relationships in a more canonical way –Provide pros and cons The single presenters can serve as indices which help the user to classify information. –Provide information from different points of view, e.g. businessman versus tourist Presentation teams can serve as rhetorical devices that allow for a continuous reinforcement of beliefs –involve pseudo-experts to increase evidence

© W. Wahlster, DFKI Presentation Teams for Advanced ECommerce I recommend you this SLX limousine.

© W. Wahlster, DFKI Underlying Knowledge Base Representation of domain –FACT attribute car_1 consumption_car_1 Value dimensions for cars adopted from a study of the German car market –safety, economy, comfort, sportiness, prestige, family and environmental friendliness –FACT polarity consumption_car_1 economy negative Difficulty to infer implication of dimension on attribute –FACT difficulty consumption_car_1 economy low

© W. Wahlster, DFKI Example of a Dialogue Strategy Question: How much gas does it consume? Answer: It consumes 8l per 100 km. Negative Response: I’m worrying about the running costs. Dampening Counter: Forget about the costs. Think of the prestige! Header: (dampening_counter ?agent ?prop ?dim) Constraints: (*and* (positive ?agent) (pol ?prop ?other_dim positive)) Inferiors: (Speak ?agent (“Forget about the ” ?dim “!”)) (Speak ?agent (“Think of the ” ?other_dim “!”))

© W. Wahlster, DFKI But, it’s fast! Multiple Decision Support Agents... weighted propositions User-Adaptive Search Planning Spare parts for this car are rather expensive! Current and Future Work: Multiple Interface Agents for User-adaptive Decision Support

© W. Wahlster, DFKI Experimental evidence for effects of modality on the user's trust (van Mulken, 1999) The system gives recommendations, which turn out to be wrong in some cases. How much does a user trust the system's advice depending on the modality of a presentation? Self-animated Persona, Speech, Gesture, Facial Expression, Pointing Speech, Graphical Highlighting Text, Graphical Highlighting Personified Agents Increase the User's Trust in the System's Presentation

© W. Wahlster, DFKI Result: Persona > Speech > Text Conclusion: If the presentation is more human-like, recommendations are more readily followed For decision support systems tutoring systems recommendation systems virtual sales agents personified interface agents have a clear advantage: They increase the user's trust in the information presented by the system Impact of the modality of a Presentation on the User's Trustfulness

© W. Wahlster, DFKI Plug-InsApplets Add features (character players) to browser Download triggered by user Requires disk space on client Unrestricted access to client Less appropriate for WebCommerce, Guides Agents integrated in 3D environments Appropriate for Entertainment Examples: Extempo's Jennifer James (Hayes-Roth et al. 98) PFMagic's virtual petz Java animation code sent over the net Automatic loading Requires no disk space on client Restricted access to client Appropriate for WebCommerce, Guides Agents integrated in 2D environments Less appropriate for Entertainment Examples: DFKI's Web Persona (Müller et al. 98) ISI's Adele (Johnson et al 98) New in AiA/PAN: Balanced combination of Applets and Servelets Efficient distribution of client-side Java and server-side Java for driving the Interface Agent Sending Interface Agents to Clients: Plug-Ins or Applets?

© W. Wahlster, DFKI Intelligent Web Services Personalized Interface Agents User Modeling Planning Natural Language Understanding Knowledge Representation Image Understanding Machine Learning Plan Recognition Information Retrieval Multimodal User Interfaces Research on Personalized Interface Agents brings disparate subfields in the area of intelligent systems together

© W. Wahlster, DFKI ECommerce projects of DFKI have shown that research on personalized interface agents can be transferred to real world applications: Dekra (largest European organization of used car dealers): FairCar as an ECommerce platform with NL access and a comparison shopping agent for used cars DaimlerChrysler: online user modelling in a one-to-one marketing system for Mercedes cars Otto/Shopping24/Eddie Bauer (largest European tele-order company): virtual sales agents for one-to-one marketing of fashion and computer hardware Porsche: Virtual Market for Pre-owned Porsche Cars Conclusion

© W. Wahlster, DFKI The generation of virtual webpages with personalized interface agents leads to innovative applications in: Electronic Commerce, Electronic TV Guides (EPG) Telelearning environments, Call Centers and Help Desks Two Research Challenges: Making the Interface agents sensitive to temporary limitations of the user‘s time and working memory capacity (cf. our READY project, Jameson et al., p in IUI99 Proceedings) Making the agents instructible, so that they can learn from the user in a dialog, how to extract information in a new domain (cf. Mathias Bauer, Dietmar Dengler TrIAs: Trainable Information Assistants for Cooperative Problem Solving in Agents'99, on Tuesday) Conclusion