University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Global Scale Tele-Immersion Network Performance Activities Jason Leigh, Oliver Yu, Linda Winkler, Alan Verlo, Tom DeFanti Yong-joo Cho, Ray Fang, Javier Girado, Liujia Hu, Tomoko Imai, Naveen Krishnaprasad, Michael Lewis, Ya Ju Lin, Dave Pape, Kyoung Park, Chris Scharver, Brenda Silva, Liang Wang Josh Eliason, Jinghua Ge, Eric He, Atul Nayak, Shalini Venkatamaran

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Common Characteristics of Teleimmersive Applications

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Characterization of Tele-Immersive Streams Estimated bandwidth (bits/s) DiffServ TypesBurstiness Latency sensitive Jitter sensitive Error sensitive UDP avatar 6K x n (15fps) Interactive Real-time ConstantYYN UDP audio stream 64K x nBriefYYN UDP video stream 10M (2-way only) ConstantYYYN UDP stream With Playback depends Non- interactive Real-time ConstantYNYN TCP control data 7K x nReliableBriefYN Y TCP bulk data depends Best Effort or Deadline Delivery Sustained burst NNY

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Network Research

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Maximizing Bandwidth Utilization over Long Fat Networks Even if QoS via DiffServ or IntServ is available, it still does not solve the Long Fat Network problem Problem is small TCP window sizes (well known problem but still no widely accepted solution) On SGIs change in window size requires kernel rebuild Size of window should be set to current available BW of the network CAVERNsofts Parallel Socket Striping works well but is considered irresponsible use of networks

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) 64K Window Size Amsterdam to Chicago Bursty as max bw reached but performance is still good

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) 64K Window Size CERN to EVL Bursty as max bw reached but performance still good

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Window Size: EVL = 1.85M, SARA = 64K SARA to EVL EVL to SARA Sending client determines the window size When window size is large enough no real benefit to using parallel sockets

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Window size: EVL = 1.85M, CERN = 640K CERN to EVL EVL to CERN Similar story at CERN

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Anomalies Theoretical BW from EVL to SARA is 100Mbps Netperf UDP shows reasonable performance: –EVL to SARA 85Mbps –SARA to EVL 65Mbps (5 more hops via Abilene) Netperf and Parallel sockets TCP shows only: –30Mbps Perhaps due to asymmetric tcp window size settings? Argument for UDP-based schemes? E.g. Forward Error Correction

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Forward Error Correction scheme for low- latency delivery of error sensitive data Transmit error correction data over high bandwidth networks that can be used for correcting UDP streams to achieve lower latency than TCP but higher reliability. Transmit error correction data to improve quality of streamed video by correcting for lost packets. Not intended for bulk data transfer but in light of TCP results this might hold some promise.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) FEC Experiments EVL to SARA- Amsterdam (45Mb/s 100ms RT latency) Broader Ques: –Can FEC provide a benefit? How much? –Tradeoff between redundancy and benefit? Specific Ques: –TCP vs UDP vs FEC/UDP –How much jitter does FEC introduce? –High thru put UDP vs FEC/UDP to observe loss & recovery

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) ` FEC greatest benefit is in small packets. Larger packets impose greater overhead. As redundancy decreases FEC approaches UDP. goal

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) G o a l

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Packet Loss over UDP vs FEC/UDP between Chicago & Amsterdam Data Rate (bits/s) Packet Size (Bytes) Packet Loss Rate in UDP (%) Packet Loss Rate in FEC over UDP (%) 1M M M M M M UDP FEC

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Human Factors in Tele-Immersion

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Collaborative Coordination Experiments between Chicago and Singapore CAVE to CAVE (STAR TAP) Audio via Phone call Scramnet (adjustable latency, 0 jitter) LAN Ethernet (~ 10ms) Local ISDN (~ 200ms) STAR TAP (~ 250ms) Predict STAR TAP similar to performance over ISDN

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Collaborative Coordination Experiments between Chicago and Singapore 200ms RTT is the threshold where performance begins to suffer Roughly RTT to Asia. Results to Singapore similar to local ISDN 200ms RTT with 0 jitter is same as 10ms RTT with 7ms jitter

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) 25Mbps 80Mbps ANL EVL 42Mbps 100Mbps fore back Bandwidth recovery good Latency recovery good Small packet loss DiffServ Experiment 1 + background+ DiffServ x x xx

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) DiffServ Experiment 2 25Mbps 80Mbps ANL EVL 42Mbps 100Mbps fore back Bandwidth recovery good Latency recovery not good Packet loss double + background+ DiffServ x x xx

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Application of Research Results CAVERNsoft G2 applications at iGrid 2000 in Yokohama

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Tele-Immersion Middleware The CAVERNsoft G2 Toolkit G2 is C++ toolkit for building Tele-Immersive applications with special emphasis on networking Networking: –UDP, TCP, Multicast, HTTP. –UDP reflector and multicast bridge. –TCP reflector. –Remote procedure calls. –32 and 64bit Remote file I/O. –Parallel 32 & 64 bit TCP socket striping for high throughput data delivery. –FEC library. –Client/Server distributed shared memory persistent database. –Threading, Mutual Exclusion. –Built-in Instrumentation of networking services. –QoS via GARA and MCSP underway.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Tele-Immersion Middleware The CAVERNsoft G2 Toolkit Audio streaming. Articulated Avatars. VR navigation. VR menus. Speech recognition with IBM ViaVoice. Collaborative application shell to jumpstart development.

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) TIDE Teleimmersive Data Explorer (TIDE) In collaboration with National Center for Data Mining General framework for collaborative visualization of massive data-sets Current data-set is ozone data from NOAA

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) CIBRView Collaborative Image Based Rendering Viewer (CIBRview) In collaboration with Wes Bethel and Steve Lau at Lawrence Berkeley Lab Accesses volume data 512x256x256x 256 frames ~ 40Gig data-sets Generates image slices that are distributed to collaborating clients. Sent about 500 slices/files from Chicago to Japan

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Virtual Harlem University of Missouri Virtual Harlem Reconstruction of Harlem during the Harlem Renaissance

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Earthquake Hypocenters Space Physics & Aeronomy Research Collaboratory (U of Michigan) A demonstration at Telecom 2000 and SC 2000 between Israel, Dallas, Chicago, Michigan

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Network Visualization

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) QoS Internet Monitoring Tool QoSIMoto

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) STAR TAP Network Visualization

University of Illinois at Chicago Electronic Visualization Laboratory (EVL) Future Work DiffServ and RSVP from EVL to CERN in collaboration with NWU Reliable UDP for high throughput bulk data transmission Integrated Collaboratory for Analysing Networks (iCAN): iCAN-Monitor, iCAN-Visualize, iCAN-Manage, iCAN-Active Test, iCAN-Collaborate