Realizing High Quality Digital Video Over Internet July 16th 2008 TEIN2 NOC Workshop Kazunori Sugiura ( uhyo@kmd.keio.ac.jp )

Digitized Video Transport System How can we transport live multi-media contents?

Hardware Media Format MPEG-2 H.264 DV Raw-HD WMV MOV RM WAV MP3 Ogg メディアコンテンツとハードウェアが無限にある雰囲気 Ogg Analog TXT Meta-Data

New Generation Internet: GLOBAL COMPUTING ENVIRONMENT To fulfill the IT requirements of every social and living activities in Asia, This project aims new generation Internet infrastructure by inter-connecting every activities. Digital Broadcast Contents on Demand Metro, Regional Transportation Aerospace and Satellite Sea, and Air Transportation Social Services Houses, Apartments, Buildings Cars, Bikes Education Society Global Computing Environment in Asia Core Technology for New Asia Internet Digital Media and Contents Life style and Livings Social and Public Activities

Once upon a time

Streaming ?! 1996 Internet World Exposition

Rally Raid Mongolia 1996 Internet World Expo

Frame rate and Resolution fps (frames per second) NTSC：30fps (29.97fps) PAL：25fps Films：24fps Resolutions ( dot resolutions) 30 Picts 30picts. 30fps 30fps 1 Second 1 Second

Quality and Contents By means of media contents Quality ( SD to QHD ) Bandwidth ( 1.5M to 4.5G up) Encoding Characteristics ( High Computation, Delays…)

We are here DVD Digital BS 802.11g 802.11n HDDVD We are Blu-ray here 100BaseT DVD DV World (720x480) 1000BT TCP Friendly IPv4/IPv6 802.11n Embedded Digital BS HDV World (MPEG2 720p 1080i) Multicast HDDVD Blu-ray FEC We are here HD World (MPEG2，H.264 1080p) DRM IPTV 10G NGN Digital Cinema (MJ 4096x2160, 2048x1080) Real HD World Uncompressed (1920 x 1080 p) QHD (3840x2160) 10G Stack 16HD(SHD) (7680x4320) 64HD(VSHD) (15360x8640)

Resolution and Bandwidth Pixel Dot 32Bit 1Frame 30fps byte Bandwidth bps 640x480 307,200 1200K 35.16M 281M 800x600 480,000 1875K 54.93M 439M 1024x768 786,432 3M 90M 720M 1280x1024 1,310,720 5M 150M 1200M 1600x1200 1,920,000 7.32M 219.6M 1.72G 1920x1080 2,073,600 7.91M 237.3M 1.85G 1920x1200 2,304,000 8.78M 263.4M 2.06G 3960x2400 9,504,000 36.25M 1087.5M 8.50G

Summary Media streaming over Internet is a challenge Bandwidth Computational power Resolution and video quality Advances in media compression technique Advances in packet transport technology

Compression Technique

Compression Technique Frame based Compression Method Motion JPEG DV Inter Frame Compression Method MPEG2 MPEG4 H.264

JPEG/Motion JPEG JPEG Motion JPEG Lossy data compression picture format DCT(Discrete Cosine Transform) algorithm Cutting High frequency Motion JPEG Combining JPEG pictures to make frame based video

Compress even more Using higher compression method Motion based lossy compression No movement background Movement Target Using the recently used Inter Frame compression

MPEG Video CODEC Data standard Data handling MPEG1 MPEG2 (MPEG3 is gone ) MPEG4 (H.264) Data standard MPEG7 Data handling MPEG21

MPEG1 Specifications MPEG1 uses SIF Format Bandwidth: 1.5Mbps Standards for Video CD MPEG1 uses SIF Format Non Interlace 360×240×29.97Hz(NTSC)

GOP(Group Of Picture) structure Frame combination I picture (Intra picture) P picture (Predictive picture) B picture (Bi-directional predictive picture) Forward prediction Bi-directional prediction I Pictire B Pictire P Pictire 15Picture/1GOP B B I B B P B B P B B P B B P

MPEG2 Higher quality encoding compared to MPEG1 Examples Broadcast Quality Examples DVD HDV Digital Broadcast

Resolutions for MPEG2 MP@HL 1920×1152 ×60 4:2:0 HP@HL 4:2:2 MP@H-14 1440×1152 Spt@H-14 HP@H-14 SP@ML 720×576×30 Bピクチャなし MP@ML SNR@ML HP@ML MP@LL 352×288×30 SNR@LL

MPEG4 Higher Compression method than MPEG2 1998 – Object based encoding For low bandwidth MPEG-4 AVC (Part 10 ) H.264

DV DV(Digital Video) 35.382Mbps DV Cassette mini-DV Cassette IEC61834 (1999) Resolution：720x480(NTSC) 25.146Mbps Audio 1.536Mbps 48kHz/16bit 2 channel 32kHz/12bit 4 channel Frame Compression using DCT DV Cassette mini-DV Cassette

HDV Canon, Sharp, Sony, Victor (2003) Resolution 1280x720 (720p) 19.7Mbps 1440x1080 (1080i) 25Mbps 1080/25p 1080/30p 1080/24p Audio：48kHz/16bit 2 channel MPEG1 Audio Layer2 (384Kbps) MPEG2 Inter Frame Compression DV, mini-DV

DVCPRO Panasonic (1996) DVCPRO Cassette DVCPRO DVCPRO 50 DVCPRO P 480i(NTSC) 25Mbps DVCPRO 50 480i(NTSC) 50Mbps DVCPRO P 480p(EDTV-II) 50Mbps DVCPRO HD 1080i/720p 100MBps Interoperability with DV Format DVCPRO Cassette

What is Streaming? Streaming Download Playback Receive Data while playing Does not store data as a file ( DRM (Digital Rights Management) perspective Download Playback Download contents as a file, and playback afterwards Internet Internet Download Playback Streaming

Streaming Technology To absorb characteristics of the Internet Some of the software technologies Buffering ( Delays) Session Management Multiple Bit-rate ( Multiple Quality) Bandwidth reservation, QoS Error Correction Contents Protection

History in Streaming 1994 1995 1996 1997 Real Audio1.0 StreamWorks1.0(First Commercial streaming applications) VIC, VAT, RAT developed by UCL (University College London) 1995 Real Audio1.0 1996 Internet World Exposition IWE’96 RTP(RFC1889) 1997 MS NetShow2.0

History in Streaming (contd.) 1999 QuickTime4.0(ストリーミングへ対応) VLC (Video Lan Client) DVTS 2002 Helix( Open source ) 2003 Windows Media 9

Streaming Theory

TCP and streaming TCP Using TCP for streaming Retransmission method Packet receiving order guarantee Congestion Management Using TCP for streaming How can we manage the real-time when Retransmission occurs? Congestion Management happens?

UDP Streaming UDP No Retransmission No Packet order guarantee No Congestion Management To maintain some sort of packet transport guarantee to End-toEnd RTP/RTCP

RTP/RTCP RTP Real-time Transport Protocol RTCP Real-time Transport Control Protocol Standardized in RFC1890 Standardization to transport real-time data through the network

RTP Real-time Transport Protocol RFC1889 Standard protocol for streaming Common information required to send real-time data Sequence Number Timestamp Dedicate RFC for data dependent part Many payload format dependent on their compression method and media formats RTP itself does not reserve resources or manage and guarantee the QoS

RTP Packet v p X CC Payload Type Sequence Number Time Stamp Extension CC CSRC Count Marker Payload Type Sequence Number Time Stamp Synchronization Source (SSRC) Contribution Source (CSRC)

RTP related RFCs RTP Basic Specifications RFC1889 RTP: A Transport Protocol for Real-Time Applications. RFC1890 RTP Profile for Audio and Video Conferences with Minimal Control. Other Payload Specifications RFC2198 RTP Payload for Redundant Audio Data. RFC2793 RTP Payload for Text Conversation. RFC2833 RTP Payload for DTMF Digits, Telephony Tones and Telephony Signals. RTP Generals RFC2508 Compressing IP/UDP/RTP Headers for Low-Speed Serial Links.

RTP Payload Format Payload Format RFC2029 Sun's CellB Video Encoding. H.261 Video Streams. RFC2035 JPEG-compressed Video. RFC2038 MPEG1/MPEG2 Video. RFC2190 H.263 Video Streams. RFC2250 RFC2343 Bundled MPEG EXPERIMENTAL RFC2429 the 1998 Version of ITU-T Rec. H.263 Video (H.263+). RFC2431 BT.656 Video Encoding. RFC2435 RFC2658 PureVoice(tm) Audio. RFC2862 Real-Time Pointers. RFC3016 MPEG-4 Audio/Visual Streams. RFC3047 ITU-T Recommendation G.722.1.

RTP Payload Type PCMU A 8000 1 1016 2 G.721 3 GSM 5 DV14 6 16000 7 LPC Encoding Audio / Video Frequency (Hz) Channel (音声) PCMU A 8000 1 1016 2 G.721 3 GSM 5 DV14 6 16000 7 LPC 8 PCMA 9 G.722 10 L16 44100 11 14 MPA 90000 15 G.728 25 CelB V 28 Nv 31 H.261

RTCP(Realtime Control Protocol) RFC1889 To Control RTP Packet flow control, clock transmission between sender and receiver Quality report by receiver Jitter Lost packet RTCP Sender Report Reports to receiver or the 3rd application for status report on sending conditions

RTCP Packet Report block comes afterwards v=2 P RC Length Payload Type SR=200 Length Header Sender Synchronization Source SSRC NTP Timestamp Fore 32bit(Send time for reports) NTP Timestamp Lower 32bit RTP Timestamp Sender Info Sender Packet count Sender Octet Count Report block comes afterwards

RTCP Method RTCP Receiver Report BYE APP Packet Loss rate, Sum of packet losses, Sequence number of received packet, jitter, last timestamp of sender report (LSR), and Delay between LSR (DLSR) BYE APP Application Extension

RTCP Method SDES Receiver information LOC CNAME TOOL NAME NOTE EMAIL Receiver identifier NAME Receiver name EMAIL Mail address PHONE Phone No. LOC Location TOOL Application name NOTE User condition PRIV Application extension

Summary Streaming application over Internet uses UDP / IP RTP RTCP Ensure real-time isochronous transmission

DVTS Design

Video format used in IEEE1394 DV Format A.K.A “DV Camera” Frame base compression 30Mbps 720 x 480 Pixels (NTSC) 720 x 576 Pixels (PAL) HDV Format MPEG2-TS(PS) compressed High Definition DV Inter Frame compression ( = Latency ) Somewhat different from “HDTV” 1080i (1440 x 1080)25Mbps, (1920 x 1080 ) 720p (1280 x 720) 19Mbps Less Bandwidth compared to DV 30 > 25

Simple Mechanism Packet Losses What happens when packet losses occur: Reuse the past frame data Past frame Past frame data Is used Packet losses Reused frame Present frame

Frame rate reduction and Bandwidth in DV Format (Mbps) Frame Rate

Network Friendly streaming Packet shaping and burst-less transmission Packets time

Shaping packets VIDEO VIDEO VIDEO AUDIO AUDIO AUDIO AUDIO AUDIO AUDIO BURSTY PACKETS BURSTY PACKETS BURSTY PACKETS VIDEO VIDEO VIDEO AUDIO AUDIO AUDIO AUDIO AUDIO AUDIO Fragmented Video Frames AUDIO AUDIO AUDIO AUDIO AUDIO AUDIO

Packet Flow Control

Congestion mechanism Gateway / Router Drop Drop Drop

End application based packet shaping (Avoiding burst traffic) Large packet buffer pool

Jitter caused by priority mismatch Audio packet Sender host Application Video packet Waiting for audio packet Waiting for audio packet Waiting for audio packet

Separation of packet Audio Packet Sender host Video packet Application Audio queue Video queue Priority audio packet

TCP＋UDP Traffic TCP sender packets TCP packet UDP packet congestion UDP sender packets

TCP＋UDP Traffic TCP sender packets TCP packet UDP packet congestion UDP sender packets

Decrease packets to avoid congestion TCP＋UDP Traffic TCP sender packets TCP packet Decrease packets to avoid congestion UDP packet UDP sender packets

TCP＋UDP Traffic TCP sender packets TCP packet Decrease packets to avoid congestion UDP packet UDP sender Rate will not change (No congestion avoidance) packets

TCP＋UDP Traffic TCP sender packets TCP packet UDP packet UDP sender Decrease packets to avoid congestion UDP packet UDP sender Rate will not change (No congestion avoidance) packets

summary DVTS is developed with software technology to Adapt into various condition of network Try to be friendly with other packet / networks Avoiding bursty packets Using TCP Friendly algorithms Using past packet / loss of packets

DVTS Update

DVTS supports HDV streaming Support Video Format MPEG2-TS 1080i and 720p Support Operating-System Linux (kernel 2.4.27 or later and 2.6.x) Newest Libraw1394 driver is required. Windows XP VLC Transmit IEEE1394 Output Display Recording DVTS for Linux DVTS for Windows Video LAN Client

DVTSng Rev.2 Merge DVTS and HiDVTS applications on one package DV -> use DV mode application JVC HDV -> use HDV-JVC mode application Sony HDV -> use HDV-SONY mode application IPv6 multicast (ASM/SSM) update Rev.1 cannot use IPv6 multicast function getaddrinfo() doesn’t run -> re-enable Download URL http://beta.dvts.info/setup-0.0.0-2.exe このURLはmuscat.sfc.wide.ad.jpのalias このままでもいいですし，適宜移動してもOKです other device will be supported on next revision

Not support IEEE1394 output DV Mode Application Support IEEE1394 output Not support IEEE1394 output Please use HDVout tool instead!! HDV Mode Application

HiDVTS / Camera Output Tool Functions HDV(MPEG2) RTP data receive IPv4, IPv6 receive port unicast, multicast (ASM/SSM) IEEE1394(HDV device) output only support SONY device (maybe) Download URL http://beta.dvts.info/hdvout-setup-1.0.0-1.exe

Select HDV device (camera/VCR) Start/Stop running Select IP version Quit application If you want to use multicast, Specify multicast address, interface and source address (SSM)

DVTS supports Multicast subsets (including IGMPv3 and MLDv2) Linux kernel 2.6.x supports v4/v6 SSM Windows XP supports only v4 SSM(IGMPv3) Vista supports v4/v6 SSM(IGMPv3,MLDv2) IGMPv3 MLDv2 DVTS for Linux DVTS for Vista DVTS for Windows XP Video LAN Client

MacOS Update gldvshow was released DVTS1.0g was released No Audio Support FEC Support OpenGL error monitor Support DVTS1.0g was released Support MacOS 10.5 Leopard

cbrmaker(1/3) Constant Bit Rate UDP stream making stream in Application Layer You can check Bandwidth Transmission delay Packet arrival interval time Packet loss You can get http://www.sfc.wide.ad.jp/dvts/software/cbrmaker/ Support platform Linux i386 only I think everybody has gone through for unknown reasons packet loss.

cbrmaker(2/3) You can draw graphs from logs of cbrmaker Transmission delay Packet arrival interval time Packet loss

cbrmaker(3/3) Parameter Example of the utility bps (bit per second) pps(packet per second) Packet size usleep time between packets Example of the utility OS / network driver test Network bandwidth test Network quality test

Global Studio

Global Studio Location(2008) NEW(June,2007) Stanford U. US

Changes on existing studio Echo canceller installation To provide better audio environment Schedule Cambridge(September 07) Yonsei(March 08) Tsinguha(March 08) Replacing video transmission PCs To reduce audio noise More CPU power(for HD level video transmission) Updating　equipments at Mita campus Improving audio and video quality Digital Audio for noiseless audio environment HD video Collaboration with SOI-Asia

The Stanford Studio Installed in June 2007 Location Mobile Equipments Collaboration with Stanford university. Location The Wallenberg Hall Mobile Equipments 1st floor learning theater or other conference room

Mobile Equipment cabinet Mobile cabinet(outside) Handle to carry the cabinet Removable doors Wheels Mobile cabinet(inside) Polycom EF2241 Audio echo canceller Cisco Catalyst 2960 Audio Patch panel(All XLR) DVTS PCs(Laptop PC is used after 2007)

DMC Studio @Keio Mita Location: Tokyo, Japan Operated by: Keio University DMC Institute DVTS and Polycom / Multipoint capable IPv4/IPv6

Studio at Tsinghua University Location: Beijing, China Operated by: Tsinghua University Time difference: 1 hour

Studio at Yonsei University Location: Seoul, Korea Operated by: Yonsei University Connecting to KOREN (Korea Advanced Research Network) Established in February 2006

Studio in San Francisco Location: California, USA 15 minutes drive from SFO Airport in Data Center Stanford University Time difference: 17 hours

Studio at the University of Cambridge Location: Cambridge, UK Operated by: University of Cambridge Time-zone: -9 hour (JST)

Studio in New York Location: New York, USA Operated by: Japan Society near United Nations HQ Operated by: Japan Society Time difference: 13 hours

Configuration Plan for Cambridge Studio AC TAP Polycom VSX 7000S Panel Monitor ADVC-110 DVTS/HDVTS S L Systems WIN-XP HD Note-PC Projector Audio Mixer Behringer FBQ2496 POSTERS Behringer MX882 Video Light Network Camera(SD) MIC 20-182　B-U Converter Ethernet Switch 8 to 16 port Giga UPLINK DVTS/HDVTS S L Systems WIDE Conversion Lenz IEEE1394 Long Cable Sony HVR-Z1J Ethernet RGB / DVI Headphone ADVC-110 Tripod DVTS/HDVTS S L Systems Accessory Kit IEEE1394 S-Video Audio Version 1.0 uhyo

DMC Manifesto

Closing session at Mita Mozilla 24 24 hour World Wide continuous event Connects Mozilla community member on the earth for collaboration. Different Time zone Distributed location DMC provide the “Global Studio” as communication environments. DVTS based conference Real video streaming Date & Location September 15th -16th Japan/US/France SOI-ASIA www.mozilla24.com Discussion at Paris Closing session at Mita

Cool Japan Support NHK TV program “Cool Japan” In November 2007 Provide DVTS video conference environment San Francisco, Cambridge, Seoul and Mita On Air December 12th DVD is available Contact kudo Received video at Cambridge Received video at Mita

Other Events The 6th DMC symposium Keio/Kyoto MoU ceremony etc Connect Beijing for Prof. Murai December 2007 Keio/Kyoto MoU ceremony Connect Mita and Kyoto September 2007 etc

Feedback and Evolve 社会 Society デザイン DESIGN マネジメント MANAGEMENT 体験EXPERIENCE 企画 IDEA ポリシー POLICY デザイン DESIGN マネジメント MANAGEMENT テクノロジ 　TECHNOLOGY 製品化PRODUCT 試作PROTOTYPE 市場 Market 検証TESTBED 91

SIGCOMM2007 IPv4/v6 dual stack Multicast Live Streaming using Consumer HD Video

outline SIGCOMM2007 live streaming Inter-Domain Multicast Routing Status DVTS for HDV(MPEG2-TS HD) Next step to deploy IPv6 Multicast

SIGCOMM 2007 Live Streaming WIDE Project(AS2500) provided Internet connectivity for SIGCOMM2007 participates. We also provided 2 workshops and 3 conferences live streaming to world wide researchers who could receive. We prepared following streams. IPv4 and IPv6 MPEG2-TS 1080i streaming to world wide researchers. IPv6 MPEG4 streaming to TEIN2 and SOI-ASIA members.

Inter-Domain Multicast Routing Some Asian Academic AS have been enabled IDMR since SIGCOMM2007.  Tein2-SG and Tein2-HK has been connected to Tein2-JP. Koren has been connected to APAN-JP. Good content makes us promote IDMR. World Wide IDMR ring was constructed! Most of Academic AS have multiple AS-Path.

World WIDE Multi-Home IDMR MAP GEANT2 TransPAC2 APAN-JP Abilene TEIN2-JP TEIN2-HK TEIN2-SG 実際に受信しているのを確認した AS の図 v4 版 ルータプロキシ、オペレータ仲間を利用して調べました 実際の受信サイト全部じゃありません、もっと居ると思います （特に tein2-HK / SG の先) Koren は v4 only で、玄海を通るパスは SIGCOMM によって開通 ちなみに Tein2 SG / HK の v4/v6 も SIGCOMM によって開通だったかも（ここクリアにして明日渡します） AARNET3

IPv4 Listener AS (Confirmed at SIGCOMM2007) WIDE KOREN TransPAC2 APAN-JP Abilene TEIN2-JP TEIN2-HK TEIN2-SG 実際に受信しているのを確認した AS の図 v4 版 ルータプロキシ、オペレータ仲間を利用して調べました 実際の受信サイト全部じゃありません、もっと居ると思います （特に tein2-HK / SG の先) Koren は v4 only で、玄海を通るパスは SIGCOMM によって開通 ちなみに Tein2 SG / HK の v4/v6 も SIGCOMM によって開通だったかも（ここクリアにして明日渡します） AARNET3

IPv6 Listener AS (Confirmed at SIGCOMM2007) UNINETT NORDNET GEANT2 WIDE TransPAC2 APAN-JP Abilene TEIN2-JP TEIN2-HK TEIN2-SG IPv6 版です IPv6 の方が顕著ですが、世界中の学術ネットワークを参加させるのは時間の問題 もうこれからマルチキャストはビジネスの世界に落ちていっても良い時代かもしれない やっぱり人気コンテンツって大事だよね（sigcommえらい） 何はともあれ！ APAN / Tein がアジア・ユーラシアのハブ局になっているのがよく分かる 不安定と言われ続けてきたマルチキャストを、各リーフから集めて世界中にばらまく役目を担っている 評価されるべきだよね APAN と Internet2 の会議に殴り込みに行って v6 multicast を通して貰った甲斐がありました 各サイトの SIGCOMM がはじまってからの素早い対応を褒めたい

SIGCOMM2007 ASIA IPv6 Multicast Live Streaming 13 Mbps UDL RP DVTS 30 Mbps MPEG4 VideoLAN 1Mbps SIGCOMM2007 @ Kyoto, Japan AI3/SOI Asia @Keio, Japan Uninet members APAN-JP Easy collaboration because of small time-differences (0 ~ 3hr) High demand on the international cooperation in higher education. High demand on the internet deployment High value of educational sharing among multi-lateral and multi-culture countries. MPEG4 VideoLAN 1 and 3Mbps ThaiREN/Uninet @Thailand 24 SOI Asia partner Universities @ 12 Asia countries TEIN2-JP RP INHERENT members TEIN2-SG INHERENT @Indonesia

Backbone is ready! and then… How to deploy IPv6 multicast? Backbone is ready to provide IPv4 and IPv6 multicast. BUT! Users don’t care of IP version, they want to receive good “contents”. We have to prepare the IPv4/v6 transparent multicast environment for the users. In addition, IPv6 has a priority to use IPv4:p

DVTS supports HDV streaming Support Video Format MPEG2-TS 1080i and 720p Support Operating-System Linux (kernel 2.4.27 or later and 2.6.x) Newest Libraw1394 driver is required. Windows XP VLC Transmit IEEE1394 Output Display Recording DVTS for Linux DVTS for Windows Video LAN Client

Minimal Settings Just prepare 1 Camera and 2 PCs. Optional: HDV Deck. IEEE1394 UDP/RTP HDV Camera $2,000 - $10,000 IP Network HDV VCR IEEE1394

DVTS supports Multicast subsets (including IGMPv3 and MLDv2) Linux kernel 2.6.x supports v4/v6 SSM Windows XP supports only v4 SSM(IGMPv3) Vista supports v4/v6 SSM(IGMPv3,MLDv2) IGMPv3 MLDv2 DVTS for Linux DVTS for Vista DVTS for Windows XP Video LAN Client

SIGCOMM2007 Live Settings Canon XL H1 Roland v-440 HD SONY HVR-Z1J Component/1080i Component/1080i Speaker’s Laptop RGB-15pin Component/1080i Component/1080i Audio Mixer Roland VC300-HD Canon Hall PA Roland VC300-HD Canon IEEE1394(HDV) IEEE1394(DV) IPv4/v6 Multicast To Global Multicast Cloud hdvsend for Linux dvsend for Linux DV RTP To SFC ADVC-110 Composite IPv6 Multicast To Tein2 via AI3 IEEE1394(DV)

MBGP AS-Path Table from UNINETT NORWAY Connect from APAN-JP MBGP AS-Path Table from UNINETT NORWAY AS2500 WIDE 左上　APAN-JP のセグメントで見ている東大の学生 右　　　UNINETT(ノルウェー）のASから見た WIDE のマルチキャスト的位置 もちろん一つ上は APAN-JP です 左下 30Mbps * 2 の dual stack multicast が流れてるねーのトラフィック図 AS7660 APAN-JP Live Streaming Equipments

SIGCOMM2007 ASIA IPv6 Multicast Live Streaming Easy collaboration because of small time-differences (0 ~ 3hr) High demand on the international cooperation in higher education. High demand on the internet deployment High value of educational sharing among multi-lateral and multi-culture countries.

Hi vision and beyond

i-Visto 1.5Gbps非圧縮HDTVをはじめとする各種の高品質映像信号を、IPネットワーク上の複数の拠点間でリアルタイム伝送するシステム。HD-SDI等をIPに変換し送受信する装置（Gateway）や各種ビットレートの映像ストリームをリアルタイムに蓄積・配信する映像サーバ（eXmedia server）から構成。 i-Visto media converter i-Visto camera HDTV SDTV IP network (LAN/WAN) HD-SDI i-Visto eXmedia server HDTV equipment SD-SDI SDTV equipment i-Visto manager i-Visto gateway XG HDTV monitor i-Visto product lineup

Traffic(cont.) HTB(10GE) Sapporo NW center(10GE) Incomming from Sapporo NW center (v6-multi) Sapporo NW center(10GE) Outgoing to KDDI Sapporo (v4-uni) KDDI Sapporo(1GE*2) Outgoing to KDDI Sendai (v4-uni)

Traffic(cont.) ABC(10GE) Incoming (v4-uni) Outgoing (v6-multi) NTT Otemachi(10GE) Forwarding (v4-uni&v6-multi)

i-Visto(cont.) Packetize each line Configuration Not each frame Configuration Resolution：1080i or 760p Color：8bit(1Gbps) or 10bit(1.6Gbps) Network interface: 2 * 1GE Usually use 10GE NIC Over subscribe 1GE*2 circuit Segmentation for each 1GE Testing IPv6 Multicast send/receive at 1470byte(most suitable frame size) Pre-Test Packet loss occurred by lack of performance at receiving side

Layered Multicast

Layered multicast for adaptive modulation Problem statement With WiMAX multicast, multicast data transmission rate depends on the link capacity of the node whose link quality is worst Nodes with good wireless connection cannot get high-quality multicast data Combination of layered multicast and adaptive modulation Nodes come to be able to receive the multicast data with proper quality depends on their wireless connection quality Key Technolgies Layered multicast Network resource aware multicast data transmission method Multicast group management Dynamic IP multicast group management based on the nodes’ current physical layer modulation Mapping between an IP multicast group to a modulation based WiMAX multicast group

Layered Multicast Hierarchical data structure Multicast tree structure 10/10 10 Layered Multicast 8/10 8 Hierarchical data structure Data are divided into several “Layer” Layer is multimedia data unit Video quality control selecting the number of accepting layers Layer Base Layer Base Multimedia data Enhanced Layer Quality enhancement data Multicast tree structure Data are deliverd with IP multicast Senders provide maximum quality data Intermediate nodes decrease layers to send adapting to the network resource 6/10 6 6/10 6 6/10 6 1/10 1 Base Enhanced 1 Enhanced 2 Enhanced 3 Enhanced 4 Enhanced n

WiMAX electric wave method 64QAM ready 16QAM read QPSK ready BPSK ready

Modulation based multicast group management image 64QAM + 16QAM + QPSK + BPSK zone 16QAM + QPSK + BPSK zone BS QPSK + BPSK zone BPSK zone

Modulation based multicast group management Enhanced 1 + 2 + 3 64QAM (Enhanced 3) Base + Enhanced 1 + 2 16QAM (Enhanced 2) Signal type Base + Enhanced 1 QPSK (Enhanced 1) BPSK (Base Layer) Base only distance

Research-Slide

Research background: The growth of mixed environment User requirement User requirement High quality Event relay download Live Network Collaboration game Service characteristic VoIP IPTV Digital Cinema Service or user-tool Unicast Multicast P2P Resource management Priority control authentication security End control End control Network characteristic AS virtual-network AS AS AS 10G 802.11a/b/g Real-network 802.16e 40,100G NGN 119

Motivation Due to the use of high speed Network(1Gbps, 10Gbps, etc), high bandwidth (high video audio quality) streaming will become common on the internet What is a requirement for high-bandwidth streaming?, and impact on other applications? Ex) What is the tradeoff point on user-oriented application? What is streaming application problem in nearly future ? What is the solution? studio studio HD Camera HD Camera

Adaptive Real-time Streaming High interactiveness Video conference Online game, 3D graphics contents Need to maintain strict packet interval time Difficult to maintain best possible quality Network condition change effect on largely Important to precisely detect a network condition change. Network associated system vs. End-to-End system The former merit: easy to quickly and precisely detect network condition (ex. Bottleneck ling) possible to adjust according to it Priority control、Bandwidth guarantee The latter merit: not need for specific router or switch implementation Loss based (or RTT-based) adaptation Network Estimation Network Controller End-Node controller Quality Adaptation

Research point (1/2) How can various streaming flows conduct best adaptation according to network condition and change?? Network estimation Various network environment Various network characteristic Various congestion The number or type of competing flows Network load User-oriented quality adaptation High network utilization Effectively consume the network resource Contents quality protection Maintaining quality for end-usage Stability Frequently quality change would degrade it’s quality User-friendliness (impact on other applications) Scalability (Congestion Control) TCP-fairness  high network utilization Various definition of fairness Network estimation Network Condition Quality adaptation (Congestion Control)

Research point (2/2) Quality adaptation Network estimation High network utilization  Contents quality protection, Scalability The way of changing transmission rate The way of maintaining quality on changeable network condition TCP-friendly could be best solution ? Network estimation Network indicator What is necessary and effectual? Need new indicator or indicator cooperation Signaling mechanism Responsiveness  accuracy Transmission Protocol Video format Vide/Audio data Application/service policy network client Media source Quality protection Network indicator Congestion Control scalability 123

Previous work Specializing in quality protection High network utilization The mechanism of minimizing packet loss Multiple control using FEC Quality adaptation Rate Control (discarding frame, so depends on video format) Dynamic FEC (Reed-Solomon Code) Network condition estimation Packet loss rate on a interval time The number of consecutive lost packets ( means a network load) FEC recovery rate Application policy network Rate Control client ユニークな点は，FECレート決定にconsecutivelossの数を利用していること，FECをnetowork estimationに利用しており，パケットエラー耐性を兼ね備えながら Quality adaptationしていくという多重制御メカニズムにあります． Media source Quality protection ・Packet loss rate ・The number of consecutive loss packets ・FEC recovery rate Congestion control Dynamic FEC

Future work Investigate “effective” adaptation mechanism Verification using a simulation Definite the mechanism tradeoff point High-network utilization, quality protection, stability and scalability Network condition (bandwidth and RTT, etc), the number and type of TCP- flow and UDP-flow R TFRC effectiveness ? How can it be change on network condition ? My algorithm effectiveness? Investigate “effective” adaptation mechanism The mechanism of knowing network condition and change. Survey Verification of detection and learning adaptation mechanism

Research-Slide Support material

The problem of quality reduction Receiver Sender Internet bursty congestion DV/RTP packet According to network condition, pktloss happens Physical bandwidth or available bandwidth Σ(DVTS traffic + other traffic ) > available bandwidth Congestion Quality reducing

The existing solution with DVTS(1/2) Reuse the past frame data Rate Control to save the consumption bandwidth Σ(DVTS traffic + other traffic ) bandwidth < available bandwidth

The existing solution with DVTS(1/2) Static FEC to packet loss recovery DV/RTP packet FEC packet FEC technology congestion FEC decode!! Packet loss recovery

The demerit of existing solutions In the condition of network congestion Reuse the past frame data and Rate Control Solve the block noise Cannot solve the reducing video quality Smoothness of frequently moving scene Static FEC Increasing the total consumption bandwidth There is the case of quality reducing more !! Users adapt the transport method by hand What is How to provide the best possible Video and Play quality ???

Motivation Dynamic Adaptation To provide the best possible quality The user doesn’t have to change the DVTS options According to the network condition change, adaptive DVTS dynamically adapt the transport method DV full rate + FEC 10% ??? DV half rate + FEC 30% ??? DV 1/3 rate + FEC 20% ???

approach Rate Control with Dynamic FEC Mechanism Frame rate control reducing bandwidth in case of fatal bandwidth conditions  To save the total consumption bandwidth Dynamically adapting FEC rate keeping play and video quality Dynamic available bandwidth proving via FEC What happens when network condition get well ?? Adapting Frame rate to current available bandwidth Providing best possible video quality sender Internet Video frame data Changing each rate FEC data

Adaptive DVTS design sender receiver Receive Report transmit report RTCP RR Packet Receive report module Report transmit module ・Network state ・Network state ・Current flow type ・before ・pktloss pattern ・Flow type Flow control module Network state measurement module FEC decode module 何を目トリックに計測を行うのか Rate control module FEC encode module Transmit Buffer RTP Packet (data, FEC) tmp buffer Play Buffer

The number of consecutively lost packet and FEC recovery rate Frec: FEC recovery rate Fenc: FEC encoding rate L: (the number of non-recoverd UDP packets within 5 seconds) L －　L’ (L > L’, L≠0) L Frec(%) = (L≦L’, L≠0)

Adaptive DVTS vs Normal DVTS BEST CASE Background bandwith:90M Anticipating available bandwidth

関連研究 Streaming Adaptation “Adjusting Forward Error Correction with Quality Scaling for streaming MPEG”, huahui NOSSDAV 2005 ACM, june A Rate Control Scheme for Adaptive Real-Time Applications in IP Networks With Lossy Links and Long Round Trip Times “A Dynamic Congestion Control Mechanism for Real Time Streams over RTP”, Ahmad, N, ICACT 2007, Feb A Rate Control Scheme for Adaptive Video Streaming over the Internet, Panagiotis, IEEE ICC 2007 proceegings ネットワーク特性/streaming品質特性に関する論文 “End-to-End Internet Packet Dynamics”, Vern Paxson LBNL-404488, june23,1997 “Analysis of Audio Packet Loss in the Internet “, INRIA B.P 93,Jean Bolot, HUgues Crepin,　Andres Vega Garda 06902 Sophia-Antipolis Cedex “Improving Accuracy in End-to-End Packet loss Measurement”, SIGCOMM2005

Network States and Flow Types in Full rate(30Mbps) Loss rate Consecutive loss 0% 10% 20% 30% F1 FEC0% F2 0<R<1 3 ≦ C < 10 FEC20% FEC10% F3 10 ≦ C FEC30% F4 1≦ R< 3 C < 10 F5 10≦ C < 25 F6 25≦ C F7 3≦ R< 8 C < 30 F8 30≦ C < 70 F9 70 ≦ C Half-FEC50% F10 8≦ R< 13 C < 100 F11 100≦ C < 170 F12 170≦ C Half-FEC60% F13 13≦ R< 18 C < 180 F14 180≦ C < 250 Half-FEC70% F15 250≦ C Half-FEC80% F16 18≦ R< 23 C < 300 F17 300≦ C < 380 Half-FEC90% F18 380 ≦ C F19 23≦ R C = any

Network States and Flow Types in Half Rate (15Mbps) Loss rate Consecutive loss 50% 60% 70% 80% 90% 100% H1 ************* FEC100% Normal-Full H2 0<R<1 3 ≦ C < 10 Full=FEC10% H3 10 ≦ C Full-FEC20% H4 1≦ R< 3 C < 10 Full-FEC30% H5 10≦ C < 25 H6 25≦ C FEC90% H7 3≦ R< 8 C < 30 H8 30≦ C < 70 H9 70 ≦ C FEC80% FEC70% FEC60% H10 8≦ R< 13 C < 100 H11 100≦ C < 170 H12 170≦ C H13 13≦ R<18 C < 180 FEC50% H14 180≦ C < 250 H15 250≦ C H16 18≦ R<23 C < 300 H17 300≦ C < 380 H18 380 ≦ C H19 23≦ R<28 C < 420 H20 420≦ C < 600 H21 600 ≦ C H22 28≦ R<33 C < 750 H23 750≦ C < 1000 H24 33≦ R C < 1300 H25 1300 ≦ C

G8 summit operation using FEC DVTS

Overview Hokkaido Receiving each G8 national broadcast kanagawa Encoder FEC DVTS Receiver kanagawa FEC DVTS Receiver converter FEC DVTS Sender DV streaming with FEC 30% converter FEC DVTS Sender

Backbone SNet / JGN2plus T-LEX Cisco2.notemachi NTT Business Ether * 4 Vlan 3750 203.178.138.129 Gi 7/13-16 Vlan 3758 203.178.139.97 SNet / JGN2plus T-LEX Cisco2.notemachi NTT Business Ether * 4 1Gbps 10Gbps 100Mbps * 4 Foundry6.otemachi 本牧ハマーズ Net 203.178.139.96/27 札幌 IMC Net 203.178.138.128/28 Foundry4.nezu Nec2.yagami Cisco2.fujisawa L2 Gi 2/1/2 203.178.139.65 L3 SFC ITC L2 Backbone 100M 1Gbps * 1 SFC λ館 Net 203.178.139.64/28 1G 10G 141

Hammers in kanagawa Cisco2.notemachi Untag vlan 3758 203.178.139.97/27 Gi7/13 Gi7/16 Gi7/14 Gi7/15 Gi1/0/1 Gi2/0/1 Fa0/1 Fa0/1 Cat3750-1 Cat3750-2 Cat2950-1 Cat2950-2 BBC CNNI TV-5 DWTV RAITV RTR NHK World YOBI

Sapporo IMC Cisco2.notemachi Te 3/4 Vlan 3750 203.178.138.129 T-LEX catalyst3750.sapporo tag vlan 3750 203.178.138.142 1/0/1 1/0/2 1/0/15 1/0/14 1/0/3 1/0/7 1/0/5 1/0/13 1/0/4 1/0/6 BBC-R CNNI-R TV-5-R DWTV-R RAI-TV-R RTR-R NHK-World-R YOBI1 YOBI2 YOBI3

SFC (backup stream) ax2430.sfc-ramda 0/22 0/23 TV5-S DWTV Cisco2.fujisawa Gi 2/1/2 203.178.139.65 203.178.139.78 ax2430.sfc-ramda 0/22 0/23 TV5-S DWTV

System overview in Sapporo NHKメディコン後 プレビュー NHKエンコーダ 直前プレビュー WIDE NHK

WIDE Rack NTSC DVモニタ (WIDE) PAL DVモニタ (NHK) port assign and IP address 1/0/1:203.178.138.130:BBC-R 1/0/2:203.178.138.131:CNNI-R 1/0/3:203.178.138.132:TV-5-R 1/0/4:203.178.138.133:DWTV-R 1/0/5:203.178.138.134:RAI-TV-R 1/0/6:203.178.138.135:RPR-R 1/0/7:203.178.138.136:NHK-World-R 1/0/13:203.178.138.137:YOBI-1 1/0/14:203.178.138.138:YOBI-2 1/0/15:203.178.138.139:YOBI-3 BBC CNN TV5 YOBI-2 YOBI-3 DW RAI RTR NHK YOBI-1 工具・シリアル NHKスイッチ(to留寿都) WIDEスイッチ(toJGN)

Monitor Preview in Sapporo