© 2005 Avaya Inc. All rights reserved. A Client-Side Architecture for Supporting Pervasive Enterprise Communications Amogh Kavimandan, Reinhard Klemm, Ajita John, Doree Seligmann, Aniruddha Gokhale Vanderbilt University, Nashville TN Avaya Labs Research, Basking Ridge NJ

2 © 2005 Avaya Inc. All rights reserved. Hermes: Pervasive, Context-Aware Enterprise Communications Middleware Hermes Enterprise Communications Middleware Business processes need to communicate with people and connect people with each other Hermes automates people communications by enabling design and execution of communications applications Integration with business processes Hermes communications applications connect the right people, at the right time, via the right communication media Communications Application Business Process Communications Application Communications Application Bob Alice Cary Conferencing Systems Communication Servers Messaging Systems Voice Switches

3 © 2005 Avaya Inc. All rights reserved. launches Expert Finder Application Example of Hermes Communications Application Conference Bridge Manager Conference Hermes Alice in her home office Browser Hermes User Portal logs into bridges in sends dialog pops up returns filled dialog transmits user context Client-Side Agent Pervasive, context-aware enterprise communications middleware requires client-side support –Specific to communication endpoints, media fills out dialog

4 © 2005 Avaya Inc. All rights reserved. Client-Side Challenges for Pervasive, Context- Aware Enterprise Communications (1/3) Pervasive, context-aware communications applications make decisions about who, when, how to contact/connect people Effective decision-making requires efficient collection of user context: –User presence: which communication endpoint(s) is user connected to? –User communication activity: what communication activities has user engaged in during last x time units? With whom/what is user communicating right now?

5 © 2005 Avaya Inc. All rights reserved. Client-Side Challenges for Pervasive, Context- Aware Enterprise Communications (2/3) –User interruptibility: can user be interrupted with request for feedback? Computed from user presence, communication activity, nature of request, enterprise and personal rules/policies –User availability: can user participate in new communication session? If not now, how soon? Computed from user feedback, for specific communication session –Communication device characteristics: can user participate in/continue with new/ongoing communication using current communication endpoint? Static and dynamic characteristics

6 © 2005 Avaya Inc. All rights reserved. Client-Side Challenges for Pervasive, Context- Aware Enterprise Communications (3/3) –Support for application-user dialogs: dialogs for collecting user availability feedback either outside or during ongoing communication session Notifying user of pending dialogs or barge-in Rendering (multiple) dialogs Collecting user responses Dialog presence –Session management: users may be engaged in several communications sessions simultaneously and need to receive dialogs in the right session(s) –Transparency: user should not perceive difference in performing various communication tasks despite ongoing user context collection

7 © 2005 Avaya Inc. All rights reserved. Argus: A Client-side Architecture for Supporting Pervasive Enterprise Communications (1/2) Argus complements enterprise communications middleware on the client-side for supporting pervasive, context-aware communications applications Negotiation between users and communications applications through Argus agents Enterprise Communications Middleware Endpoint 1 Endpoint 2 Endpoint 3 Endpoint 4 Argus Agent Session Mgmt. Device Monitor Session Mgmt. Device Monitor Session Mgmt. Device Monitor Session Mgmt. Device Monitor

8 © 2005 Avaya Inc. All rights reserved. Argus: A Client-side Architecture for Supporting Pervasive Enterprise Communications (2/2) Argus agents are deployed with user communication endpoints Agents provide context gathering, propagation, dialog presentation functionalities Device monitor tracks and reports device properties of interest – e.g. battery remaining, interaction capabilities etc. Session management associates session information with individual communication endpoints

9 © 2005 Avaya Inc. All rights reserved. Web Browsers as Communication Endpoints Web-based user context data collection is meaningful since employees spend a lot of time interacting with enterprise Web portals that expose a variety of enterprise applications Web browsers are universally deployed, programmable client software of choice for many user activities Can render multimedia contents easily Already provide infrastructure support for uploading information, rendering interaction (through forms) and returning user feedback Firefox Web browser extension mechanism for implementing data collection functionality Other communication endpoints to follow…

10 © 2005 Avaya Inc. All rights reserved. Argus Architecture Argus consists of two main components: –Communication Agent: Tightly coupled with endpoint (here: browser). Responsible for user context gathering, dialog presentation, feedback collection –Machine Agent: Mediates between user and middleware. Maintains session information, sends device characteristics to middleware One Communication Agent per communication endpoint, but only one Machine Agent per client machine Web Services Client Proxy Server Device Agent Per endpoint instance Machine Agent Per device User Device Enterprise Communication Middleware Web Services Client Proxy Server Device Agent Communication Agent AJAX Components User Response UI Elements Activity Monitor

11 © 2005 Avaya Inc. All rights reserved. Communication Agent Communication Agent monitors user activity and manages dialogs –Activity Monitor Context data may be the current page being presented, time spent on the page, what Web application is being used etc. Periodically packages observed context data and sends package to Machine Agent –User Response Component Makes upcalls into the AJAX components to forward the context data Receives dialogs to be rendered as forms to the user –AJAX Components Asynchronous components Create server requests from JavaScript calls, return control back to extension event loop

12 © 2005 Avaya Inc. All rights reserved. Machine Agent Single instance of Machine Agent per device provides session management for all active sessions –Proxy Server Single point of contact for Communication Agents to send user context information to Maintains session information, dialog graphs –Device Agent Currently implemented for Windows XP but can be easily ported to other operating systems Collects, reports static/dynamic device capabilities (device context) Device context helps enterprise communications middleware to adapt dialog design

13 © 2005 Avaya Inc. All rights reserved. Dialog Rendering and Injection Synchronized Multimedia Integration Language (SMIL), designed for describing and rendering interactive multimedia presentations, is used for dialogs –Standardized development tools –Rendering supported in Web browsers Dialogs are associated with session ID of target browser session Machine Agent detects and receives (through Java/Web Services calls) list of pending dialogs for active session from middleware Communication Agent retrieves dialogs (as SMIL scripts) through HTTP requests and renders them through User Response component in Firefox extension Web Services Client Proxy Server Device Agent Machine Agent User Device Enterprise Communication Middleware Web Services Client Proxy Server Device Agent Communication Agent AJAX Components User Response UI Elements Activity Monitor JS CallsHTML HTTP XML Java Method Calls Web Services Calls

14 © 2005 Avaya Inc. All rights reserved. User Perspective (1/2) Communication Agent implemented as Firefox extension: –Context gathering is transparent – no perceivable change in browsing experience –User can turn on/off context gathering at any point Dialogs are rendered in the form of a pop-up window which the user can fill in –May contain audio, video

15 © 2005 Avaya Inc. All rights reserved. User Perspective (2/2) Proxy server Communication Agent Dialogs may be presented piecemeal, where each dialog depends on the outcome of the previous user response Can be seen as a graph maintained at Machine Agent

16 © 2005 Avaya Inc. All rights reserved. Conclusion and Future Work We presented the client-side architecture Argus for supporting pervasive, context-aware enterprise communications Argus gathers user context information, propagates it to enterprise communications middleware, and supports multimodal dialogs, thereby accelerating decision-making in enterprise communications applications We plan to do several user studies for quantifying usability and measure accuracy of user context data provided by Argus Argus has been extended to work with Thunderbird client; we are also looking at extending it to a range of communication endpoints