1. Integration of Hand-Held Devices into Collaboration Environments IC’02 Las Vegas, NV June 27 2002 June 27 2002 Geoffrey Fox, Sung-Hoon Ko, Kangseok Kim, Sangmi Lee, Sangyoon Oh Community Grids Laboratory, Indiana University Computer Science, Indiana University Computer Science, Florida State University

2. Contents Garnet Collaborative system Garnet Collaborative system Universal Accessibility in Garnet Universal Accessibility in Garnet Integration Handheld devices into collaboration environment Integration Handheld devices into collaboration environment Shared Applications in Garnet for Universal Accessibility Shared Applications in Garnet for Universal Accessibility Experimental Result Experimental Result Conclusions and Future Works Conclusions and Future Works

3. Collaborative Systems Collaborative Systems supports coordination and cooperation of two or more people who attempt to perform a task or solve a problem together. Collaborative Systems supports coordination and cooperation of two or more people who attempt to perform a task or solve a problem together. Collaborating directly, sharing resources. Collaborating directly, sharing resources. The collaboration features: The collaboration features: Basic interactive tools - Text Chat, Instant Messenger, White Board Basic interactive tools - Text Chat, Instant Messenger, White Board Shared resources – Shared Display, Resource Exporting, Sharing I/O devices Shared resources – Shared Display, Resource Exporting, Sharing I/O devices Real-time Multimedia tools- Audio-video Conferencing Real-time Multimedia tools- Audio-video Conferencing

4. Garnet Collaborative System Purpose : Support distance Education, Training and if possible Computing as Grid(Web) Services Purpose : Support distance Education, Training and if possible Computing as Grid(Web) Services Integrate Synchronous and Asynchronous collaboration Integrate Synchronous and Asynchronous collaboration Support universal access including PDA ’ s collaboration with desktops Support universal access including PDA ’ s collaboration with desktops Uniform XML event (message) based architecture Uniform XML event (message) based architecture All data structures defined in XML Schema GXOS All data structures defined in XML Schema GXOS XML for all metadata (Users, documents, computers) and object changes -- from text chats to display changes etc. XML for all metadata (Users, documents, computers) and object changes -- from text chats to display changes etc. MyXoS manipulate GXOS objects MyXoS manipulate GXOS objects We build on GMS/JMS (Java Message Service) as industry standard to implement publish/subscribe model We build on GMS/JMS (Java Message Service) as industry standard to implement publish/subscribe model Support collaborative features : basic interactive features (textchat, whiteboard, etc.), shared resources (shared display, shared export), AV conferences. Support collaborative features : basic interactive features (textchat, whiteboard, etc.), shared resources (shared display, shared export), AV conferences.

5. GMS (Garnet Message Service) System All message publication labels and subscription profiles are defined in XML All message publication labels and subscription profiles are defined in XML Publishers Subscribers Message Queue Labeled by (XML) Topic Object Subscriber Profile Objects Specify Query to Event Label Database Subscribes to all events to get persistence

6. Integration Handheld devices into collaboration environment Garnet provides universal access including PDA’s collaborating with desktops Garnet provides universal access including PDA’s collaborating with desktops GMS is not able to support light-weight clients. GMS is not able to support light-weight clients. Limited computing power – CPU, memory size, display Limited computing power – CPU, memory size, display Network Environment – unreliable, unstable wireless communication, various network protocols Network Environment – unreliable, unstable wireless communication, various network protocols Developing Environment – various operating systems Developing Environment – various operating systems New universal collaboration and access architecture is required: GMSME (Garnet Message Service Micro Edition) New universal collaboration and access architecture is required: GMSME (Garnet Message Service Micro Edition)

7. GMSME Supports heterogeneous clients by customizing message depending on the client specifications. Supports heterogeneous clients by customizing message depending on the client specifications. User profile and mobile device profile (screen size, device type) is defined with XML object metadata specification (GXOS) User profile and mobile device profile (screen size, device type) is defined with XML object metadata specification (GXOS) Consists of PDA Adaptor, Hand Held Message Service (HHMS) protocol, and API for application processors. Consists of PDA Adaptor, Hand Held Message Service (HHMS) protocol, and API for application processors.

8. GMSME – PDA Adaptor PC PDA Adaptor PC PDA Adaptor PDA Adaptor looks like a conventional client to GMS and adapts data to specified client/user specifications PDA Adaptor looks like a conventional client to GMS and adapts data to specified client/user specifications “Listens” to all the messages and events on the Garnet collaboration systems by subscribing to GMS. “Listens” to all the messages and events on the Garnet collaboration systems by subscribing to GMS. PDA Adaptor PDA Client PDA Adaptor PDA Client Parses the GMS XML message body and delivers it to the mobile clients over HHMS. Parses the GMS XML message body and delivers it to the mobile clients over HHMS. HHMS is a specially designed optimized protocol for light- weight devices. HHMS is a specially designed optimized protocol for light- weight devices.

9. PDA Adaptor (Personal Server)

10. GMSME – Application Transports Instant Messenger Instant Messenger supports to access from PDA Jabber clients to Jabber server. supports to access from PDA Jabber clients to Jabber server. Parsing XML messages between PDAs and Jabber Server. Parsing XML messages between PDAs and Jabber Server. Managing Presence information based on XML presence protocol (XXMP). Managing Presence information based on XML presence protocol (XXMP). Text Chat Text Chat extends Text Chat feature built in Garnet to Mobile clients. extends Text Chat feature built in Garnet to Mobile clients. Translates GMS messages between PDAs and Garnet collaborative systems. Translates GMS messages between PDAs and Garnet collaborative systems.

11. IM and Text Chat Transports

12. GMSME – Application Processors Shared Display Shared Display Sharing frame-buffer corresponding to either a windows or entire desktop replicated among the clients. Sharing frame-buffer corresponding to either a windows or entire desktop replicated among the clients. customizable display resolution ( smaller resolution, different display types etc.) customizable display resolution ( smaller resolution, different display types etc.) Data compression, incremental update. Data compression, incremental update. Shared Export Shared Export Convert (rendering of) object to some standard form that is more flexible than bitmap of Shared Display. Convert (rendering of) object to some standard form that is more flexible than bitmap of Shared Display. Replicated appropriate viewer for each client (PDF HTML SVG Browsers) Replicated appropriate viewer for each client (PDF HTML SVG Browsers) Adapts W3C’s SVG (Scalable Vector Graphics) for exporting 2D graphics. - Inherits the benefits of Vector graphics, XML document based model, SVG universal accessibility features (stylesheets, alternatives). Adapts W3C’s SVG (Scalable Vector Graphics) for exporting 2D graphics. - Inherits the benefits of Vector graphics, XML document based model, SVG universal accessibility features (stylesheets, alternatives).

13. The Architecture of Shared Export The Architecture of Shared Export The workflow of shared SVG Export in Garnet System Well Formed SVG Document A Well Formed SVG Document AAA Convert to SVG Convert to SVG Power Point Slides Power Point Slides HHMP JMS/GMS Server GMS JMS/GMS Server GMS Grid Shared SVG Browser Grid Shared SVG Browser Grid Shared SVG Browser GMS Grid Shared SVG Browser Grid Shared SVG Browser Grid Shared SVG Browser Grid Shared SVG Browser Grid Shared SVG Browser GMS Personal Server Parser Rendering Engine Style sheets Personal Server Parser Rendering Engine Style sheets

14. My_Professor Collaborative User Interface providing integrated GUI to Garnet users accessing from mobile devices. Collaborative User Interface providing integrated GUI to Garnet users accessing from mobile devices. Implements HHMS client for wireless devices. Implements HHMS client for wireless devices. Provides user information about device, user, application to PDA adaptor. Provides user information about device, user, application to PDA adaptor. Designed specially for various devices : network protocols, display types, etc. Designed specially for various devices : network protocols, display types, etc. Delivers messages from PDA Adapter to each application modules; Delivers messages from PDA Adapter to each application modules; Display IM message converted to HHMS by PDA Adaptor. Display IM message converted to HHMS by PDA Adaptor. Draw ready-to–use image for Shared Display and Export. Draw ready-to–use image for Shared Display and Export.

15. Garnet : Shared Display PCs

16. Garnet : SVG PCs

17. Experimental Results GMSME : iPaq H3650, WinCE 3.0, Personal-Java1.1 GMSME : iPaq H3650, WinCE 3.0, Personal-Java1.1 Wireless 11 Mbit/s IEEE 802.11b Wireless 11 Mbit/s IEEE 802.11b Total Latency

18. Conclusions and Future Works A collaborative environment Garnet designed a universal event model using the pub-sub paradigm. A collaborative environment Garnet designed a universal event model using the pub-sub paradigm. A integration of PDAs into synchronous collaboration. A integration of PDAs into synchronous collaboration. PDA Adaptor maps system events into a form suitable for the PDAs and uses an optimized protocol. PDA Adaptor maps system events into a form suitable for the PDAs and uses an optimized protocol. Future Works Future Works supporting various Handheld device including PalmOS and Cellular Phone. supporting various Handheld device including PalmOS and Cellular Phone. Web Service for Universal Access Collaboration Web Service for Universal Access Collaboration