1. Multimedia Networking r Application classes m streamed stored audio/video m one-to-many (multicast) streaming of real-time a/v m real-time interactive audio/video r Typical application issues m packet jitter m packet loss / recovery r Internet protocols for multimedia m RTSP m RTP/RTCP m H.323 r Text: Kurose-Ross, Chapter 6

2. Example Multimedia Apps r Streamed stored audio/video m movies, CS-653 taped lectures (available on MANIC) r One-to-many streaming m News broadcasts, popular movies r Real-Time Interactive m IP telephony, teleconference, distributed gaming

3. Multimedia terminology r Multimedia session : a session that contains several media types m e.g., a movie containing both audio & video r Continuous-media session: a session whose information must be transmitted “continually” m e.g., audio, video, but not text (unless ticker-tape) r Streaming: application usage of data during its transmission Data stream Playback pt Rcv pt

4. Multimedia vs. Raw Data r Multimedia m e.g., Audio/Video m Tolerates some packet loss m Packets have timed playout reqmts r Raw Data m e.g., FTP, web page, telnet m Lost packets must be recovered m Timing: faster delivery always preferred Why not just use TCP for multimedia traffic?

5. Jitter r The Internet makes no guarantees about time of delivery of a packet r Consider an IP telephony session: Speaker Listener Time Hi There, What’s up? Hi The t’s up? re, Wha

6. Jitter (cont’d) r Desired time-gap: S i+1 - S i Received time-gap: R i+1 - R i r Jitter between packets i and i+1: (R i+1 - R i ) - (S i+1 - S i ) Pkt i Pkt i+1 SiSi S i+1 RiRi R i+1 Sender: Pkt i+1 Pkt i Time Receiver: r A packet pair’s jitter is the difference between the transmission time gap and the receive time gap jitter

7. Buffering: A Remedy to Jitter r Delay playout of received packet i until time S i + C (C is some constant) r How to choose value for C?  Bigger jitter  need bigger C  Small C: more likely that R i > S i + C  missed deadline m Big C: requires more packets to be buffered increased delay prior to playout m Application timing reqmts might limit C: Interactive apps (IP telephony) can’t impose large playout delays (e.g., the international call effect) non-interactive: more tolerant of delays, but still not infinite...

8. Adaptive Playout r For some applications, the playout delay need not be fixed r e.g., [Ramjee 1994] / p. 430 in Kurose-Ross m Speech has talk-spurts w/ large periods of silence m Can make small variations in length of silence periods w/o user noticing m Can re-adjust playout delay in between spurts to current network conditions

9. Packet Loss / Recovery r Problem: Internet might lose / excessively delay packets making them unusable for the session Pkt i Pkt i+1 Pkt i+3 arrival time: app deadline: ii+1i+2i+3 usage status: …, i used, i+1 late, i+2 lost, i+3 used,... r Solution step 1: Design app to tolerate some loss r Solution step 2: Design techniques to recover some lost packets within application’s time limits

10. Applications that tolerate some loss r Techniques are medium-specific and influence the coding strategy used (beyond scope of course) m Video: e.g., MPEG m Audio: e.g., GSM, G.729, G.723, replacing missing pkts w/ white-noise, etc. r Note: loss tolerance is a secondary issue in multimedia coding design r Primary issue: compression

11. Reducing loss w/in time bounds r Problem: packets must be recovered prior to application deadline r Solution 1: extend deadline, buffer @ rcvr, use ARQ m Recall: unacceptable for many apps (e.g., interactive) r Solution 2: Forward Error Correction (FEC) m Send “repair” before a loss is reported m Simplest FEC: transmit redundant copies Pkt i Pkt i+1 Pkt i+2 Sender: Receiver: Pkt i Pkt i+1 duplicate i+2 lost

12. More advanced FEC techniques r FEC often used at the bit-level to repair corrupt/missing bits (i.e., in the data-link layer) r Here, we will consider using FEC at the packet layer (special repair packets): header data FEC bits Data 1 Data 2Data 3 FEC 1 FEC 2

13. A Simple XOR code r For low packet loss rates (e.g. 5%), sending duplicates is expensive (wastes bandwidth) r XOR code m XOR a group of data pkts together to produce repair pkt m Transmit data + XOR: can recover 1 lost pkt 10101  11100 00111 1100010110   10101  111000011111000 10110  

14. Reed-Solomon Codes r Based on simple linear algebra m can solve for n unknowns with n equations m each data pkt represents a value m Sender and receiver know which “equation” is in which pkt (i.e., information in header) m Rcvr can reconstruct n data pkts from any set of n data + repair pkts m In other words, send n data pkts + k repair packets, then if no more than any k pkts are lost, then all data can be recovered r In practice  To reduce computation, linear algebra is performed over fields that differ from the usual 

15. Reed Solomon Example over  r Pkts 1,2,3 are data (Data1, Data2, and Data3) r Pkts 4,5 are linear combos of data r Assume 1-5 transmitted, pkts 1 & 3 are lost: m Data1 = (2 * Pkt 5 - 3 * Pkt 4 + Pkt 2) m Data2 = Pkt 2 m Data3 = (2 * Pkt 4 - Pkt 5 - Pkt 2) Pkt 1: Data1 Pkt 2: Data2 Pkt 3: Data3 Pkt 4: Data1 + Data2 + 2 Data3 Pkt 5: 2 Data1 + Data2 + 3 Data3

16. Using FEC for continuous-media r Divide data pkts into blocks r Send FEC repair pkts after corresponding data block r Rcvr decodes and supplies data to app before block i deadline Data 1 block i D2 blk i D3 blk i FEC 1 blk i F2 blk i D1 blk i+1 Sender: Rcvr: D1 blk i D3 blk i F1 blk i F2 blk i D1 blk i+1 Rcvr App: Block i needed by app Decoder D1 blk i D2 blk i D3 blk i...

17. FEC via variable encodings r Packet contents: m high quality version of frame k m low quality version of frame k-c (c is a constant) m If packet i containing high quality frame k is lost, then can use packet i+c with low quality frame k in place i low: k-c high: ki+1 low: k-c+1 high: k+1i+2 low: k-c+2 high: k+2

18. FEC tradeoff r FEC reduces channel efficiency: m Available Bandwidth: B m Fraction of pkts that are FEC: f m Max data-rate (barring pkt loss): B (1-f) r Need to be careful how much FEC is used!!!

19. Bursty Loss: r Many codecs can recover from short (1 or 2 packet) loss outages r Bursty loss (loss of many pkts in a row) creates long outages: quality deterioration more noticeab r FEC provides less benefit in a bursty loss scenario (e.g., consider 30% loss in bursts of 3) D1:iD2:iD3:iF1:iF2:i D1:i+1D2:i+1D3:i+1F1:i+1F2:i+1 Too much FECToo little FEC

20. Interleaving r To reduce effects of burstiness, reorder pkt transmissions D1D4D7D2D5D8D3D6 D1D4D8D3D6 Sender schedule Arrival schedule Playback schedule D1D3D4D6D8 r Drawback: induces buffering and playout delay

21. Multimedia Internet Protocols r We’ll look at 3: m RTP/RTCP: transport layer m RTSP: session layer for streaming media applications m H.323: session layer for conferencing applications RTP/RTCP RTSPH.323 UDPTCP UDP/ multicast

22. RTP/RTCP [RFC 1889] r Session data sent via RTP ( Real-time Transfer Protocol ) r RTP components / support: m sequence # and timestamps m unique source/session ID (SSRC or CSRC) m encryption m payload type info (codec) r Rcvr/Sender session status transmitted via RTCP ( Real-time Transfer Control Protocol ) m last sequence # rcvd from various senders m observed loss rates from various senders m observed jitter info from various senders m member information (canonical name, e-mail, etc.) m control algorithm (limits RTCP transmission rate)

23. RTP/RTCP details r All of a session’s RTP/RTCP packets are sent to the same multicast group (by all participants) m All RTP pkts sent to some even-numbered port, 2p m All RTCP pkts sent to port 2p+1 r Only data senders send RTP packets r All participants (senders/rcvrs) send RTCP packets

24. RTP header r Why do most (all) multimedia apps require m sequence #? m timestamp? m (unique) Sync Source ID? r Why should every pkt carry the 7-bit payload type? m Why not just when sender initiates session? r Transmission rate: application specific (no congestion control specified in RTP) Payload Type Sequence # Timestamp Synchronization Source Identifier Misc

25. RTCP packets r There are several types of RTCP packets m SR: sender report - transmission & reception stats m RR: receiver report - reception stats m SDES: Source description items m BYE: end-of-participation message m APP: application-specific functions r Typically, several RTCP packets of different types are transmitted w/in a single UDP packet

26. What RTCP provides r Info to detect colliding Synch source ID’s r Contact info (e-mail, true name) of participants r Info to count # of session participants r Reception quality of all participants r How does RTCP avoid creating congestion if all participants send RTCP packets? m consider a broadcast TV transmission

27. RTCP congestion control r A session’s aggregate RTCP bandwidth usage should be 5% of the session’s RTP bandwidth m 75% of the RTCP bandwidth goes to the receivers m 25% goes to the senders r T sender = # senders * avg RTCP pkt size.25 *.05 * RTP bandwidth r T rcvr = # receivers * avg RTCP pkt size.25 *.05 * RTP bandwidth Send at (.5 + rand(0,1)) * T : why? How does each member know # of senders, # rcvrs?

28. RTCP reconsideration r Goal: prevent RTCP bandwidth explosion if everybody joins at once m Receivers who initially join will count small # of session members r Solution when first joining: 1. Compute T, and wait random time interval 2. At end of interval, reassess # of members 3. If # of members increased, compute a new T’ 4. If T’ < T, send immediately 5. If T’ >= T, wait an additional T’, go to step 2 r Other times, use normal wait period

29. RTSP [RFC 2326] r RTSP: out-of-band protocol used to control transmission of a media-stream m VCR-like functionality (pause/resume, FF, rewind, reposition, etc.) Web Browser Media Player Web Server Media Server http get presentation description teardown setup ACK play ACK pause ACK media stream ACK HTTP protocol RTSP protocol

30. H.323 r A standard for real-time audio / video teleconferncing on the Internet r Network Components: m end points: end-host H.323-compliant app m gateways: interface between H.323-compliant communication and prior technology (e.g. phone network) m gatekeepers: provide services at gateway (e.g., address translation, billing, authorization, etc.) Audio Apps Video Apps System Control G.711 G.722 G.729 etc. H.261 H.263 etc. RTP / RTCP RAS Channel H.225 Gateway Call Signaling Channel Q.931 Call Control Channel H.245 H.323 UDP TCP

31. H.323 cont’d r H.225: notifies gatekeepers of session initiation r Q.931: signalling protocol for establishing and terminating calls r H.245: out-of-band protocol negotiates the audio/video codecs used during a session (TCP) G.711 G.722 G.729 etc. H.261 H.263 etc. RTP / RTCP RAS Channel H.225 Call Signaling Channel Q.931 Call Control Channel H.245 H.323