GlobalMMCS DS-RT 2005 Tutorial IEEE DS-RT 2005 Montreal Canada Oct Geoffrey Fox CTO Anabas Corporation and Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN

Collaboration and Web Services Collaboration has a)Mechanism to set up members (people, devices) of a “collaborative sessions” b)Shared generic tools such as text chat, white boards, audio- video conferencing c)Shared applications such as Web Pages, PowerPoint, Visualization, maps, (medical) instruments …. b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment We can port objects to Web Services and build a general approach for making Web services collaborative a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we should make it a WS

Shared Event Collaboration All collaboration is about sharing events defining state changes Audio/Video conferencing shares events specifying in compressed form audio or video Shared display shares events corresponding to change in pixels of a frame buffer Instant Messengers share updates to text message streams Microsoft events for shared PowerPoint (file replicated between clients) as in Access Grid Finite State Change NOT Finite State Machine architecture Using Web services allows one to expose update events of all kinds as message streams Need publish/subscribe approach to share messages (NB) plus System to control “session” – who is collaborating and rules XGSP is XML protocol for controlling collaboration building on H323 and SIP

WS Display WS Viewer WS Display WS Viewer Event (Message) Service Master WS Display WS Viewer Collaboration as a WS Set up Session with XGSP Web Servic e F I U O F I R O Shared Input Port (Replicated WS) Collaboration Other Participants Web Servic e F I U O F I R O F I U O F I R O

WS Display WS Viewer WS Display WS Viewer Event (Message) Service Master WS Display WS Viewer Web Service Message Interceptor Collaboration as a WS Set up Session with XGSP Application or Content source WSDL Web Service F I U O F I R O Shared Output Port Collaboration Other Participants Text Chat Whiteboard Multiple masters

GlobalMMCS Design Principles A unified, scalable, robust “overlay” network is needed to support AV and data group communication over heterogeneous networking environments. A common AV signaling protocol has to be designed to support interactions between different AV collaboration endpoints. A core conference control mechanism is required for establishing and managing the multi-point conference. Finally, we’d like to use Web (Grid) -services to integrate collaboration communities in different technologies and to allow use of powerful services (like schedulers and meta-data catalogs) from general Grid/Web activities. XGSP defines WSDL and other interoperability specifications

Global-MMCS Community Grid This includes an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provides support for thousands of simultaneous users of collaboration services. The function of A/V media server is distributed using NaradaBrokering architecture. Media Servers mix and convert A/V streams Open XGSP MCU based on the following open source projects openh323 is basis of H323 Gateway NIST SIP stack is basis of SIP Gateway NaradaBrokering is open source messaging Java Media Framework basis of Media Servers Helix Community for Real Mediahttp:// open source release

Global-MMCS Service Architecture

XGSP Web Service MCU Architecture SIPH323 Access GridNative XGSP Admire Gateways convert to uniform XGSP Messaging High Performance (RTP) and XML/SOAP and.. Media Servers Filters Session Server XGSP-based Control NaradaBrokering All Messaging Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing NB Scales as distributed Web Services NaradaBrokering

Break up into Web Services Monolithic MCU becomes many different “Simple Services” Session Control Thumbnail “image” grabber Audio Mixer Video Mixer Codec Conversion Helix Real Streaming PDA Conversion H323/SIP Gateways As independent can replicate particular services as needed Codec conversion might require 20 services for 20 streams spread over 5 machines 1000 simultaneous users could require: 1 session controller, 1 audio mixer, 10 video mixers, 20 codec converters, 2 PDA converters and 20 NaradaBrokers Support with a stream optimized Grid Farm in the sky Future billion way “Video over IP” serving 3G Phones and home media centers/TV’s could require a lot of computing

GlobalMMCS and NaradaBrokering All communication – both control and “binary” codecs are handled by NaradaBrokering Control uses SOAP and codecs use RTP transport Each stream is regarded as a “topic” for NB Each RTP packet from this stream is regarded as an “event” for this topic Can use replay and persistency support in NB to support archiving and late clients Can build customized stream management to administer replay, and who gets what stream in what codec NaradaBrokering supports unicast and multicast Use firewall penetration and network monitoring services in NB to improve Q0S

Incorporating Support for Audio/Video Delivery into NaradaBrokering Added support for an unreliable transport protocol, UDP Implemented a fixed size (fast) topic (8 bytes). Designed a new compact event with minimum headers. Added support for legacy RTP clients (both unicast clients and multicast groups) Improved the routing algorithm to handle real-time audio and video stream delivery.

Delay (Milliseconds) Packet Number Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms NaradaBrokering-RTP JMF-RTP

Jitter (Milliseconds) Packet Number Average jitter (std. dev) for 50 video clients. NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms NaradaBrokering-RTP JMF-RTP

Polycom, Access Grid and RealVideo views of video-mixed streams using GlobalMMCS

Improved JMF Performance VICOld JMF ClientFast JMF Client 1 8% - 9%15% - 16 %6% - 7% 2 13% - 14%24% - 25 %9% - 10% 3 17% - 18%33% - 34 %15% - 16% 4 23% - 24%40% - 41%17% - 18% 5 26% - 27%46% - 47%23% -24% 6 32% - 33%51% - 52%27% - 28% 7 35% - 36%58% - 59%31% - 32% 8 40% - 41%62% - 63%34% - 35% Fraction of CPU used versus number of received streams The CIF-size video sequence from a 30-second movie with a lot of motions is streamed to the clients. Each stream is encoded in H.261, and has average bandwidth of 400Kbps and 20 fps.

MPEG-4 vs. H.261 We added MPEG4 video to Java Media Framework Higher quality and flexible video sizes including distributed pixels

Coupled Diverse Streams GlobalMMCS supports many diverse streams managed by “video system” Different audio and video codecs Shared display using video codecs (MPEG4 or H261) Motion JPEG – stream of images to and from PDA NaradaBrokering represents these and other collaborative streams just a “topics”; collaboration from multiple clients subscribing to a topic Text Chat Traditional lossless codec based shared display White boards Control streams Streams can be linked to provide composite topics eSports project linking video streams and real time annotation of any frame Can rewind and choose any frame of a real-time stream

eSports Snapshot Master Video Annotation Whiteboard Collaborative Video Annotation Whiteboard Synchronized Replay of archived video and annotation

eSports Architecture NB Replay/Archiving Service eSports Player RTSP Server/Proxy XGSP Clients Audio Video JMF Whiteboard Stremaing Gateway NB Storage Two way RTP link One way RTP link Local Storage access Communication channel Topic Replay/Archiving Service

Esports with sport movies

Integration of PDA, Cell phone and Desktop Grid Access NB Support for optimized PDA Communication

PDA Download video as images I

PDA Download video as images II

GlobalMMCS Status/Futures I 1. New Collaboration tools Shared IDL (Visualization), PowerPoint, OpenOffice (Applications need a month or so more) SVG game ( stable ) Whiteboard ( stable ) e-Sport ( prototype) Jabber IM client ( prototype) XGSP needs extension to support 2. JMF Audio/Video client ( stable) performance enhancement finished new codec ( MPEG4 finished; try H.264) support different platform ( Linux, Mac – Mac well developed but need to chase bug(s)) support NAT/firewall transparently like Skype screen codec for shared display ( prototype )

GlobalMMCS Status/Futures II 3. Replay & Archive (prototype) Replay Engine based on NaradaBroker Storage Service XGSP-RTSP gateway Extend RTSP and NaradaBrokering for Instant Replay 4. Web Server Portal ( stable) Standard calendar service ( iCalendar, vCalendar) Flexible conference management Need to package UI’s as portlets 5. Conferencing Media Processing Service ( Stable) Fix the bugs and add scheduling Support new codec ( MPEG4 ) 6. H.323 Gateway ( Stable) Import it to Linux platform

GlobalMMCS Status/Futures III 7. RealStreaming Gateway ( Stable ) Import it to Linux Support Mobile clients 8. Global-MMCS deployment & test Core performance measurements complete Test under the setting of multiple NaradaBroker and NAT/Firewall support deployment for AFRL, NASA, DOE portals test with remote sites 9. PDA Clients (prototype) 10. Improved video codec-based shared display 11. Scheduler of dynamic services sensitive to streaming bandwidth requirement as well as CPU use of codec conversion