1. Multimedia Networking

2. Goals of Today’s Lecture r Digital audio and video m Sampling, quantizing, and compressing r Multimedia applications m Streaming audio and video for playback m Live, interactive audio and video r Multimedia transfers over a best-effort network m Tolerating packet loss, delay, and jitter m Forward error correction and playout buffers r Improving the service the network offers m Marking, policing, scheduling, and admission control

3. Digital Audio r Sampling the analog signal m Sample at some fixed rate m Each sample is an arbitrary real number r Quantizing each sample m Round each sample to one of a finite number of values m Represent each sample in a fixed number of bits 4 bit representation (values 0-15)

4. Audio Examples r Speech m Sampling rate: 8000 samples/second m Sample size: 8 bits per sample m Rate: 64 kbps Compact Disc (CD) –Sampling rate: 44,100 samples/second –Sample size: 16 bits per sample –Rate: 705.6 kbps for mono, 1.411 Mbps for stereo

5. Audio Compression r Audio data requires too much bandwidth m Speech: 64 kbps is too high for a dial-up modem user m Stereo music: 1.411 Mbps exceeds most access rates r Compression to reduce the size m Remove redundancy m Remove details that human tend not to perceive r Example audio formats m Speech: GSM (13 kbps), G.729 (8 kbps), and G.723.3 (6.4 and 5.3 kbps) m Stereo music: MPEG 1 layer 3 (MP3) at 96 kbps, 128 kbps, and 160 kbps

6. Digital Video r Sampling the analog signal m Sample at some fixed rate (e.g., 24 or 30 times per sec) m Each sample is an image r Quantizing each sample m Representing an image as an array of picture elements m Each pixel is a mixture of colors (red, green, and blue) m E.g., 24 bits, with 8 bits per color

7. The 320 x 240 hand The 2272 x 1704 hand

8. Video Compression: Within an Image r Image compression m Exploit spatial redundancy (e.g., regions of same color) m Exploit aspects humans tend not to notice r Common image compression formats m Joint Pictures Expert Group (JPEG) m Graphical Interchange Format (GIF) Uncompressed: 167 KBGood quality: 46 KBPoor quality: 9 KB

9. Video Compression: Across Images r Compression across images m Exploit temporal redundancy across images r Common video compression formats m MPEG 1: CD-ROM quality video (1.5 Mbps) m MPEG 2: high-quality DVD video (3-6 Mbps) m Proprietary protocols like QuickTime and RealNetworks

10. Transferring Audio and Video Data r Simplest case: just like any other file m Audio and video data stored in a file m File downloaded using conventional protocol m Playback does not overlap with data transfer r A variety of more interesting scenarios m Live vs. pre-recorded content m Interactive vs. non-interactive m Single receiver vs. multiple receivers r Examples m Streaming audio and video data from a server m Interactive audio in a phone call

11. Streaming Stored Audio and Video r Client-server system m Server stores the audio and video files m Clients request files, play them as they download, and perform VCR-like functions (e.g., rewind and pause) r Playing data at the right time m Server divides the data into segments m … and labels each segment with timestamp or frame id m … so the client knows when to play the data r Avoiding starvation at the client m The data must arrive quickly enough m … otherwise the client cannot keep playing

12. Playout Buffer r Client buffer m Store the data as it arrives from the server m Play data for the user in a continuous fashion r Playout delay m Client typically waits a few seconds to start playing m … to allow some data to build up in the buffer m … to help tolerate some delays down the road

13. Requirements for Data Transport r Delay m Some small delay at the beginning is acceptable m E.g., start-up delays of a few seconds are okay r Jitter m Variability of packet delay within the same packet stream m Client cannot tolerate high variation if the buffer starves r Loss m Small amount of missing data does not disrupt playback m Retransmitting a lost packet might take too long anyway

14. Streaming From Web Servers r Data stored in a file m Audio: an audio file m Video: interleaving of audio and images in a single file r HTTP request-response m TCP connection between client and server m Client HTTP request and server HTTP response r Client invokes the media player m Content-type indicates the encoding m Browser launches the media player m Media player then renders the file

15. Initiating Streams from Web Servers r Avoid passing all data through the Web browser m Web server returns a meta file describing the object m Browser launches media player and passes the meta file m The player sets up its own connection to the Web server

16. Using a Streaming Server r Avoiding the use of HTTP (and perhaps TCP, too) m Web server returns a meta file describing the object m Player requests the data using a different protocol

17. TCP is Not a Good Fit r Reliable delivery m Retransmission of lost packets m … even though it may not be useful r Adapting the sending rate m Slowing down after a packet loss m … even though it may cause starvation at the client r Protocol overhead m TCP header of 20 bytes in every packet m … which is large for sending audio samples m Sending ACKs for every other packet m … which may be more feedback than needed

18. Better Ways of Transporting Data r User Datagram Protocol (UDP) m No automatic retransmission of lost packets m No automatic adaptation of sending rate m Smaller packet header r UDP leaves many things up to the application m When to transmit the data m How to encapsulate the data m Whether to retransmit lost data m Whether to adapt the sending rate m … or adapt the quality of the audio/video encoding

19. Recovering From Packet Loss r Loss is defined in a broader sense m Does a packet arrive in time for playback? m A packet that arrives late is as good as lost m Retransmission is not useful if the deadline has passed r Selective retransmission m Sometimes retransmission is acceptable m E.g., if the client has not already started playing the data m Data can be retransmitted within the time constraint

20. Forward Error Correction (FEC) r Forward error correction m Add redundant information to the packet stream m So the client can reconstruct data even after a loss r Send redundant chunk after every n chunks m E.g., extra chunk is an XOR of the other n chunks m Receiver can recover from losing a single chunk r Send low-quality version along with high quality m E.g., 13 kbps audio along with 64 kbps version m Receiver can play low quality version if the high-quality version is lost

21. Interactive Audio and Video r Two or more users interacting m Telephone call m Video conference m Video game r Strict delay constraints m Delays over 150-200 msec are very noticeable m … and delays over 400 msec are a disaster for voice r Much harder than streaming applications m Receiver cannot introduce much playout delay m Difficult if the network does not guarantee performance

22. Voice Over IP (VoIP) r Delivering phone calls over IP m Computer to computer m Analog phone to/from computer m Analog phone to analog phone r Motivations for VoIP m Cost reduction m Simplicity m Advanced applications Web-enabled call centers Collaborative white boarding Do Not Disturb, Locate Me, etc. Voicemail sent as e-mail

23. Traditional Telecom Infrastructure 7043 7040 7041 7042 External line Telephone switch Private Branch Exchange 212-8538080 Another switch Corporate/Campus Internet Corporate/Campus LAN

24. VoIP Gateways External line 7043 7040 7041 7042 PBX Corporate/Campus Internet LAN 8154 8151 8152 8153 PBX Another campus LAN IP Phone Client VoIP Gateway

25. VoIP With an Analog Phone r Adapter m Converts between analog and digital m Sends and receives data packets m Communicates with the phone in standard way

26. Skype  Niklas Zennstr ö m and Janus Friis in 2003 r Developed by KaZaA r Instant Messenger (IM) with voice support r Based on peer-to-peer (P2P) networking technology

27. Login server is the only central server (consisting of multiple machines) Both ordinary host and super nodes are Skype clients Any node with a public IP address and having sufficient resources can become a super node Skype maintains their own super nodes Skype Network Architecture

28. Data Transfer r UDP directly between the two hosts m Both hosts have public IP address m Neither host’s network blocks UDP traffic m Easy: the hosts can exchange UDP packets directly r UDP between an intermediate peer m One or both hosts with a NAT m Neither host’s network blocks UDP traffic m Solution: direct UDP packets through another node r TCP between an intermediate peer m Hosts behind NAT and UDP-restricted firewall m Solution: direct TCP connections through another node

29. Skype Data Transfer r Audio compression m Voice packets around 67 bytes m Up to 140 packets per second m Around 5 KB/sec (40 kbps) in each direction r Encryption m Data packets are encrypted in both directions m To prevent snooping on the phone call m … by someone snooping on the network m … or by the intermediate peers forwarding data

30. VoIP Quality r The application can help m Good audio compression algorithms m Avoiding hops through far-away hosts m Forward error correction m Adaptation to the available bandwidth r But, ultimately the network is a major factor m Long propagation delay? m High congestion? m Disruptions during routing changes? r Leads to an interest in Quality of Service

31. Principles for QoS Guarantees r Applications compete for bandwidth m Consider a 1 Mbps VoIP application and an FTP transfer sharing a single 1.5 Mbps link m Bursts of FTP traffic can cause congestion and losses m We want to give priority to the audio packets over FTP r Principle 1: Packet marking m Marking of packets is needed for the router m To distinguish between different classes

32. Principles for QoS Guarantees r Applications misbehave m Audio sends packets at a rate higher than 1 Mbps r Principle 2: Policing m Provide protection for one class from other classes m Ensure sources adhere to bandwidth restrictions m Marking and policing need to be done at the edge

33. Principles for QoS Guarantees r Alternative to marking and policing m Allocate fixed bandwidth to each application flow m But, this can lead to inefficient use of bandwidth m … if one of the flows does not use its allocation r Principle 3: Link scheduling m While providing isolation, it is desirable to use resources as efficiently as possible m E.g., weighted fair queuing or round-robin scheduling

34. Principles for QoS Guarantees r Cannot support traffic beyond link capacity m If total traffic exceeds capacity, you are out of luck m Degrade the service for all, or deny someone access r Principle 4: Admission control m Application flow declares its needs in advance m The network may block call if it cannot satisfy the needs

35. Quality of Service r Significant change to Internet architecture m Guaranteed service rather than best effort m Routers keeping state about the traffic r A variety of new protocols and mechanisms m Reserving resources along a path m Identifying paths with sufficient resources m Link scheduling and buffer management m Packet marking with the Type-of-Service bits m Packet classifiers to map packets to ToS classes…

36. Conclusions r Digital audio and video m Increasingly popular media on the Internet m Video on demand, VoIP, online gaming, IPTV, … r Interaction with the network m Adapt to delivering the data over a best-effort network m Adapt the network to offer better quality-of- service