Presentation is loading. Please wait.

Presentation is loading. Please wait.

N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Video over IP as a VoIP extension JongWon Kim, Ph.D. Advanced Network Conference (APAN Busan.

Similar presentations


Presentation on theme: "N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Video over IP as a VoIP extension JongWon Kim, Ph.D. Advanced Network Conference (APAN Busan."— Presentation transcript:

1 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Video over IP as a VoIP extension JongWon Kim, Ph.D. Advanced Network Conference (APAN Busan 2003) Aug 27 th, 2003 Networked Media Laboratory Dept. of Information & Communications Kwang-Ju Institute of Science & Technology (K-JIST) jongwon@kjist.ac.kr http://netmedia.kjist.ac.kr

2 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Contents  Media Delivery over IP  Network-adaptive Media Delivery oNetworking Support oNetwork Adaptation  Video over IP as a VoIP extension  Future of Video over IP - Toward Access Grid & Immersive Media

3 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Multimedia Delivery over IP

4 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Multimedia over IP Networks  Media Broadcasting  Hosted Streaming  Interactive Conferencing TN R Internet Streaming Sender Manager Proxy Receivers Conferencing

5 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Multimedia Delivery - Technologies  Efficient Media Representation Techniques:  Scalable source representation to deal with dynamically varying bandwidth  Robust source representation to deal with high error/loss rate  Dynamic and Reliable Media Delivery Techniques:  Network adaptation to map scalable and robust source represent ations onto the MM characteristic-aware transport protocols  Adaptive resource provisioning/management to guarantee the de sired QoS / multicast service  Universal Media Access with Security Techniques:  Unified integrated media support with a scalable information str ucture and a choice of efficient file format for MM  End-system and application customization with security concerns

6 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST 0% Packet Loss Comman d/ control (eg Telnet, Interactive games) Conversational voice and video Voice/video messaging Streaming audio/video Transactions (eg E-commerce, Web-browsing, E- mail access) Paging, Download s (eg FTP, still image) Fax Background (eg Usenet) 5% 100 msec1 sec10 sec100 sec Zero loss InteractiveResponsiveTimelyNon-critical Delay Multimedia Application’s QoS Requirements

7 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Media Delivery – Application vs Network Application/Source Network Network Characteristics and Policies: Rate (CBR, VBR, ABR, …), Error (random/bursty in BER/PLR/CRL), Delay/Jitter (tolerance) Content-aware Transport Application/Source Characteristic : steady/burst, real-time?, loss- resilient? Network-adaptive Delivery

8 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Network Adaptive Media Delivery - Networking Support Part -

9 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Media Delivery – Adaptation & Support

10 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Inter-operable Delivery with Protocol Support  Standard-based vs. Proprietary Protocols?  Related issue: Standardized media content representation (e.g. MPEG-X) vs. Others?

11 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST End-to-end QoS guarantee with QoS provisioning (admission control and QoS controls) & Network adaptation The Internet Economical and reliable network with Scalable multicasting & Optimized server location and load sharing including contents routing Paid services with Copyright management, authentication, billing, etc Required Network Infrastructure: Multicast, QoS, and others Content broadcast network

12 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Multicast Support for Multipoint Media Distribution  Multicast streaming?  Model (ASM, SSM, ALM/Overlay/P2P)  Reliable Multicast Transport  Mobile/Wireless Multicast  Resilient Multicasting of continuous media

13 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST QoS-network Support for Quality Media Delivery  QoS Model: IntServ, DiffServ, Traffic Engineering (MPLS), …  QoS signaling  Reservation and Allocation

14 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Toward Consolidated Support via Middleware?  Issues to be covered +++ oIdentification oDirectories, … oSecurity: Authentication, Authorization, … o…  Middleware: oSpecialized networked services that are shared by applications and users oA set of core software components that permit scaling of applications and networks  GRID Middleware  Home Networking Middleware

15 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Network Adaptive Media Delivery - Network Adaptation Part: Bandwidth (rate), loss, and delay -

16 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Media Delivery – Adaptation & Support

17 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Required Network Adaptation Functionalities  In case of Reliable Unicast with feedback oNetwork Congestion Control => Quality Adaptation: Source Rate Control / Network Rate Adaptation oError control: Error Resiliency and Error Recovery oQuality Recovery: Post-processing & Concealment oSynchronization Control & Adaptive Playout Media Pump Adaptable Source Congestion Control (TCP-friendly) Client Quality Adaptation Network Quality Recovery Error Control Synchronization Control

18 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST TCP-Friendly Congestion Control / Rate Control  TCP-Friendly Congestion Control: equation- based …  Toward unified congestion control handling: Congestion Manager (CM)  Congestion Control / Rate Control / Flow Control  Multicast congestion control?

19 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST TCP-Friendly Congestion Control / Rate Control (Cont.)  Rate Control by oSource Rate Control (on-line encoding only) oSource Rate Adaptation (or Shaping) oSmoothing – via Buffering oPacket Scheduling oPre-fetching

20 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Proactive and Reactive Error Controls  Network error controls oProactive (FEC), Reactive (ARQ), and Hybrid oDelay is key controlling factor  Network (Channel) error control vs Source error resiliency?  Quality Recovery: Error Concealment and Post- processing

21 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST SourceEncoder Network Adaptation SinkDecoder Network Adaptation Network Framing Application Framing Coding Error Detection Loss Detection Error Correction Erasure Correction CRC Sequence No. Retransmission FEC Quality Control (Layered Coding) Data Structuring (Synchronization) Resynchronization Loss Concealment Post Processing Courtesy of Dr. Noh NVW 2001 Error Control and Quality Recovery

22 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Synchronization Control and Adaptive Playout  Adaptive playout oUtilizing silence oUtilizing time-scale modification  Synchronization oIntra-media oInter-media oInter-client oTight vs Loosely-coupled

23 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Media Delivery – Network Adaptation MM Content Scalable stream in rate/error Integrated &prioritized stream Channel stream Network Source encoding Source transcoding & Framing/Multiplexing Trans-MUX & Channel coding with FEC/ARQ Transmission Feedback from Host/Application Feedback from Network

24 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Wired/Wireless Networks Layered Video EncodingReceiver/ User Network Adaptation Network Adaptation & Prioritized Packetization Video Pre- process- ing R-D Analysis Layered en- coding Target Minimum Buffer Size (Receiver Constraint) Target Minimum Bandwidth (Network Constraint) Frame Complexity (Quality Constraint) Constant Quality Rate Control (Source Rate/Error Resilience) (only if applicable) Prioritized Packeti- zation Layered RPI Network Monitoring & Feedback Handling Application Feedback Estimated Available Bandwidth, Loss/Delay Network Feedback P r i o r i t i ze d L ay ere d P a c k e t s R-D / Corruption Model Network Feedback (end-to-end) Network Adaptation (Network Filtering) R e c e i v e r S e l e c t i o n Media Delivery – Network Adaptation Network Adaptation (Network Rate/Loss/ Delay) Network Adaptation (Receiver / User Adaptation) Corrup- tion Analysis

25 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Video over IP as VoIP extension (V 2 oIP)

26 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST VoIP Extension: Voice + Video + Data  We want to achieve IP TELEPHONY (a.k.a. voice/video/data convergence): Real time voice, video and data communication over Internet Protocol (IP) networks  However, latest h igh-end VC system supports oH.263+ or H.263++ (w/ Annex U for background restoration) o384 kbit/s or 768 kbit/s oTDM/ISDN (H.320) or IP (H.323) oCIF (352x288) or CIF-Interlace 2x(352x288) oCapture and rendering typically interlace o25/30 fps, 50/60 fields per second oRound-trip delay around 400ms + oRound-trip delay for multipoint (w/ MCU) more than one second

27 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST V 2 oIP Building Blocks SIP MGCP H.323 Proxy Announcement Toolkit Multi-point Control Gatekeeper Toolkit Service Creation Gate- keepers Gateways Call Agents Announcement Player Conference Bridges Service Creation Gate- keepers Gateways  Protocol stacks (H.323, MGCP, SIP)  Switching cores (e.g. Gatekeepers, Call Agents)  Media engines (Announcement, Mixing, Gateway)  Service creation environment

28 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST MCU Multicast Audio and Video Unicast Audio and Video Decentralized Centralized POTS xDSL Cable PSTN Backend Servers (services) GK C / BE C Administrative Domain A Administrative Domain A Administrative Domain B GK A /BE A BE B /GK B H.323 Entity T A H.323 Entity T B H.323 Entity T C GK C1 GK A1 GK B1 Non-H.323 Entity T P GW BE CH Non-H.323 System Clearing House Signaling Physical Connection V 2 oIP Deployment

29 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Video Quality and Function Issues (1) Improve Video Quality and Size  Upcoming H.264 video encoding oProduct design stage now, 640x480 at full frame rate on one high-end DSP oBit rate around 384 kbit/s yields acceptable picture quality oSubjective quality significantly better than H.263++ w/ CIF at same bit rate  Variable, content-dependent capture rate oLinking with region-oriented (maybe combined with segmentation) encoding on top of variable frame skipping  Wide-screen video support oCan support video resolution like 2560x480 with single CCD camera with custom, wide-angle lens; Multiple DSPs for distributed codec is required; 1.5 – 3 Mbps  Progressive scan hardware oNo A/D and D/A losses; Better quality and good delay characteristics; 250ms round-trip delay over IP demonstrated?  Enhance error resilience with ERPS and so on

30 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Video Quality and Function Issues (2) Provide more natural, cooperative environment for multi-site conferencing  Let’s avoid transcoding and enhance user experience oCreate all representations needed in the sending endpoint and simulcast them (in line with layered multicast video) oIdeally suited for IP multicast environment; But can be with overlay (application-layer) multicast; oMay need some innovative changes in the control and mux protocols oNo transcoding artifacts whatsoever oNo additional end-to-end delay for multipoint when using IP multicast  Let’s get rid of fixed environment that discourages uncooperative use of the technology oToward flexible meeting environment without fixed camera position, fixed background, studio lighting, seating  Let’s add more functionalities while keeping user interface simple and straightforward

31 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Automatic Broadcast and Recording Note taking Remote Participation VoIP Extension: Real-time Collaboration  Bring Voice, Screen, Computation together to improve: Real- time Collaboration Vision from Microsoft oRich presence and IM oPC and phone integration oData/screen conferencing oInformation agent oEnhanced Meetings incoming communi cation to task list to web phone context 2003 Gurdeep Singh Pall from Microsoft Corporation

32 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Future of Video over IP - Access Grid & Immersive Media -

33 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST The Access Grid  Access Grid does for people what the computational Grid does for machines  Much more than teleconferencing, if possible at commodity prices  To enable groups of people to interact with Grid resources and to use the Grid technology to support group to group collaboration at a distance oDistributed Lectures and seminars oRemote participation in panel discussions oVirtual site visits meetings oComplex distributed grid based demonstrations

34 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Usage Areas  The Academic and Research, Government, Private Sectors  Making Remote Collaborations Work across Boundaries  Check “Multi-Sector Collaboration over the Access Grid” by J. T. von Hoffman (Boston Univ)”

35 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Access Grid Components of an AG Node Digital Video Digital Audio NETWORK Mixer Control Computer NTSC Video RGB Video Analog Audio Video Capture Computer Display Computer Audio Capture Computer Echo Canceller Ambient mic (tabletop) Presenter mic Presenter camera Audience camera Stream Type Max. latency Max. Jitter Min. BW Max. Loss Multi cast Text100 msN/A64 kbps0 %No Audio400 ms60 ms64 kbps5 %Yes Video400 ms30 ms256 kbps x 4 = 1 Mbps 25 % Yes Requires Multicast

36 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST AG extended toward ACE Immersive Media Supported Advanced Collaboration Environment Immersive Media Supported Advanced Collaboration Environment Access Grid 2.0 Access Grid  Enable group-to-group interactions at a distance  Improve the user experience  Enable complex multi-site visual and collaborative experiences  Build on integrated grid services architecture  Use quality but affordable digital IP based audio/video

37 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Broadband Content Requires More… Music CDs 160 Kbps Music Browsing 144 Kbps ½ Screen Video 300Kbps TV Shows Full Screen 600Kbps – 1.5Mbps Live Events High Resolution 6-8 Mbps Pay Per View Immersive Media > 100 Mbps  What kinds of broadband contents oHigh speed Internet access: WWW + Mail + Messaging oVideo focused: Basic TV + Pay TV + Personalized TV oNetworked games + Music downloads + Online gambling o…

38 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Future Broadband Contents Service & Immersive Media  Interactive collaboration - Collaborative design and engineering; Remote customer support; Distance learning  Remote visualization & Large-scale, multi-site computation and data mining  Shared virtual reality - Military and industrial team training / simulation; Multiplayer games; Virtual shopping malls (e-commerce); Mobile entertainment; Online tradeshows and conferences; Virtual Heritage  Any combination of the above

39 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Core Techniques for Immersive Media Service Integration Multi-modal Immersive Media Service Technology 3D Audio Generation & Reconstruction 3D Geometry Generation & Modeling Digital Contents Management & Protection Multi-D Haptic Generation & Reconstruction Immersive Media Compression & Integration 3D Video Synthesis & Reconstruction Immersive Media Delivery Immersive Media Storage & Query/Browsing Contents Service (Protocol & System) Immersive Media Content Adaptation Interactive User Interface

40 N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Thank You! Send Inquiries to jongwon@kjist.ac.kr jongwon@kjist.ac.kr


Download ppt "N ETWORKed M EDIA L AB. D EPT. OF I NFO. & C OMM., K-JIST Video over IP as a VoIP extension JongWon Kim, Ph.D. Advanced Network Conference (APAN Busan."

Similar presentations


Ads by Google