1. Hui Zhang, Fall 2012 1 15-441 Computer Networking Internet Video Delivery

2. Hui Zhang, Fall 2012 2 Two Methodologies of Building Networks and Applications  Methodology of building telephony network -First, understand application (interactive voice) and specify the detailed requirements of the application -Then build a special network that optimize the application and satisfy all the requirements  Methodology of building the Internet -Build a general purpose protocol stack and network with very minimal assumptions on applications -Let diverse type of applications run top of the network -When an application become popular and the performance becomes unacceptable, start to optimize the performance of the application deal with the disaster of application success later Have more constraints to fix the problem as an after-thought

3. Hui Zhang, Fall 2012 3 Examples of Internet Applications: Web  Web Initial design -HTML + browser + stateless HTTP on top of TCP/IP  Why the application performance is bad? What is the fix? (homework question)  Why the server performance is bad? What is the fix? (homework question) -Has to do with TCP teardown process -Server can only support limited # of simultaneously-open TCP connections Each TCP connection consumes resources such as TCB data structure, port # etc -A server closes the TCP connection and reclaims all data structures only after 2 MSL

4. Hui Zhang, Fall 2012 4 TCP Tearing Down Connection  Either side can initiate tear down -Send FIN signal -“I’m not going to send any more data”  Other side can continue sending data -Half open connection -Must continue to acknowledge  Acknowledging FIN -Acknowledge last sequence number + 1 AB FIN, SeqA ACK, SeqA+1 ACK Data ACK, SeqB+1 FIN, SeqB

5. Hui Zhang, Fall 2012 5 10-31-2006 Lecture 18: TCP Details5 State Diagram: TCP Connection Tear-down CLOSING CLOSE WAIT FIN WAIT-1 ESTAB TIME WAIT snd FIN CLOSE send FIN CLOSE rcv ACK of FIN LAST-ACK CLOSED FIN WAIT-2 snd ACK rcv FIN delete TCB Timeout=2msl send FIN CLOSE send ACK rcv FIN snd ACK rcv FIN rcv ACK of FIN snd ACK rcv FIN+ACK ACK Active Close Passive Close

6. Hui Zhang, Fall 2012 6 Examples of Internet Applications: Voice and Video  Telephony: bi-directional interactive voice (covered in last lecture)

7. Hui Zhang, Fall 2012 7 Outline  Multimedia requirements  Audio and Video Data  Streaming  Interactive Real-Time  Recovering from Jitter and Loss

8. Hui Zhang, Fall 2012 8 Internet Voice and Video (Continuous Media)  Typically sensitive to delay, but can sometimes tolerate packet loss (would cause glitches that can be concealed somewhat)  Three classes of applications with different requirements -Interactive Real-Time Two way communication: telephony (Skype) Multi-way conferencing: White house situation room, Google hangout -On-demand streaming Long form content: Netflix, Hulu Short form content: Youtube -Broadcast: NFL, ESPN live

9. Hui Zhang, Fall 2012 9 Dimensions of Requirements to Consider  How much signal distortion can be tolerated? -Ear is less tolerant than eye  What is the range of bandwidth? -Voice: 64 Kbps, can be even lower -Video: 200Kbps, 1Mbps, 2 Mbps, 6 Mbps  What is the tolerance of maximum delay? -Interactive Real-Time: 100 ms – 200 ms -On-demand streaming: Long form content: 10 – 60 seconds Short form content: 1-3 seconds -Broadcast: 5-10 seconds  Need to think of delay in terms of # of RTTs

10. Hui Zhang, Fall 2012 10 Audio Data  Telephone system uses 8-bit samples at 8kHz: 64kbits/s.  Further compression may be pointless given packet overhead.  Modern compression achieves equivalent perceptual quality with about 1/10 to 1/5 of the bits.  Most audio compression is performed in "blocks" of hundreds of original samples: adds latency.  Audio compression is lossy: it encodes something perceptually similar but really different from the original.

11. Hui Zhang, Fall 2012 11 Video Data  Unlike audio, video compression is essential: -Too much data to begin with, but -Compression ratios from 50 to 500  Takes advantage of spatial, temporal, and perceptual redundancy  Temporal redundancy: Each frame can be used to predict the next -> leads to data dependencies  To break dependencies, we insert "I frames" or keyframes that are independently encoded. -Allows us to start playback from middle of a file  Video data is highly structured Credit: http://www.icsi.berkeley.edu/PET/GIFS/MPEG_gop.gif Data dependency

12. Hui Zhang, Fall 2012 12 Challenges  TCP/UDP/IP suite provides best-effort, no guarantees on expectation or variance of packet delay  Streaming applications delay of 5 to 10 seconds is typical and has been acceptable, but performance deteriorate if links are congested  Real-Time Interactive requirements on delay and its jitter have been satisfied by over-provisioning (providing plenty of bandwidth), what will happen when the load increases?...

13. Hui Zhang, Fall 2012 13 15-441 Fall 2011 Multimedia13 On-Deman Streaming  Important and growing application due to reduction of storage costs, increase in high speed net access from homes, enhancements to caching  Interactive control by user (but often with long response time)  Ubiquitous on the web: -YouTube, Netflix, Vimeo -Television networks, Hollywood, etc. -Most local radio & TV stations -Virtually everywhere on websites

14. Hui Zhang, Fall 2012 14 Helper Application  Displays content, which is typically requested via a Web browser; typical functions: -Decompression -Jitter removal -Error correction: use redundant packets to be used for reconstruction of original stream -GUI for user control  Examples: -RealPlayer -Adobe Flash Player -Windows Media Player -QuickTime -DivX Web Player

15. Hui Zhang, Fall 2012 15 First Generation: HTTP Download  A simple architecture is to have the Browser request the object(s) and after their reception pass them to the player for display -No pipelining

16. Hui Zhang, Fall 2012 16 First Gen: HTTP Progressive Download (2)  Alternative: set up connection between server and player; player takes over  Web browser requests and receives a Meta File (a file describing the object) instead of receiving the file itself;  Browser launches the appropriate Player and passes it the Meta File;  Player sets up a TCP connection with Web Server and downloads or streams the file

17. Hui Zhang, Fall 2012 17 15-441 Fall 2011 Multimedia17 Meta file requests

18. Hui Zhang, Fall 2012 18 Buffering Continuous Media  Jitter = variation from ideal timing  Media delivery must have very low jitter -Video frames every 30ms or so -Audio: ultimately samples need <1ns jitter  But network packets have much more jitter that that!  Solution: buffers -Fill them with best effort -Drain them via low-latency, local access

19. Hui Zhang, Fall 2012 19 HTTP Progressive Download  With helper application doing the download, playback can start immediately...  Or after sufficient bytes are buffered  Sender sends at maximum possible rate under TCP; retransmit when error is encountered; Player uses a much larger buffer to smooth delivery rate of TCP

20. Hui Zhang, Fall 2012 20 Streaming, Buffers and Timing Time Max Buffer Duration = allowable jitter File Position Max Buffer Size Smooth Playback Time Buffer almost empty "Good" Region: smooth playback "Bad": Buffer underflows and playback stops "Bad": Buffer overrflows Buffer Duration Buffer Size

21. Hui Zhang, Fall 2012 21 Drawbacks of HTTP Progressive Download (2)  HTTP connection keeps data flowing as fast as possible to user's local buffer -May download lots of extra data if you do not watch the video -TCP file transfer can use more bandwidth than necessary  Mismatch between whole file transfer and stop/start/seek playback controls. -However: use file range requests to seek to video position  Cannot change video quality (bit rate) to adapt to network congestion

22. Hui Zhang, Fall 2012 22 Multimedia 2nd Generation: Real-Time Streaming  This gets us around HTTP, allows a choice of UDP vs. TCP and the application layer protocol can be better tailored to Streaming; many enhancements options are possible

23. Hui Zhang, Fall 2012 23 Example: Real Time Streaming Protocol (RTSP)  For user to control display: rewind, fast forward, pause, resume, etc…  Out-of-band protocol (uses two connections, one for control messages (Port 554) and one for media stream)  RFC 2326 permits use of either TCP or UDP for the control messages connection, sometimes called the RTSP Channel  As before, meta file is communicated to web browser which then launches the Player; Player sets up an RTSP connection for control messages in addition to the connection for the streaming media

24. Hui Zhang, Fall 2012 24 15-441 Fall 2011 Multimedia24 RTSP Operation

25. Hui Zhang, Fall 2012 25 15-441 Fall 2011 Multimedia25 RTSP Exchange Example C: SETUP rtsp://audio.example.com/xena/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY S: RTSP/1.0 200 1 OK Session 4231 C: PLAY rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0 (npt = normal play time) C: PAUSE rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37 C: TEARDOWN rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0 Session: 4231 S: 200 3 OK

26. Hui Zhang, Fall 2012 26 15-441 Fall 2011 Multimedia26 RTSP Media Stream  Stateful Server keeps track of client's state  Client issues Play, Pause,..., Close  Steady stream of packets -UDP - lower latency -TCP - may get through more firewalls, reliable Credit: some content adapted from Alex Zambelli

27. Hui Zhang, Fall 2012 27 15-441 Fall 2011 Multimedia27 Example 2: RTMP - Real-Time Messaging Protocol  Proprietary Adobe protocol  Runs over TCP  Manages audio, video, and other  Multiplex multiple streams over TCP connection

28. Hui Zhang, Fall 2012 28 15-441 Fall 2011 Multimedia28 Drawbacks of RTSP, RTMP  Web downloads are typically cheaper than streaming services offered by CDNs and hosting providers  Streaming often blocked by routers  UDP itself often blocked by firewalls  HTTP delivery can use ordinary proxies and caches  Conclusion: rather than adapt Internet to streaming, adapt media delivery to the Internet

29. Hui Zhang, Fall 2012 29 3rd Generation: HTTP Streaming  Other terms for similar concepts: Adaptive Streaming, Smooth Streaming, HTTP Chunking  Client-centric architecture with stateful client and stateless server -Standard server: Web servers -Standard Protocol: HTTP -Session state and logic maintained at server  Video is broken into multiple chunks  Chunks begin with keyframe so independent of other chunks  A series of HTTP progressive downloads of chunks  Playing chunks in sequence gives seamless video

30. Hui Zhang, Fall 2012 30 Adaptive Bit Rate with HTTP Streaming  Encode video at different levels of quality/bandwidth  Client can adapt by requesting different sized chunks  Chunks of different bit rates must be synchronized -All encodings have the same chunk boundaries and all chunks start with keyframes, so you can make smooth splices to chunks of higher or lower bit rates

31. Hui Zhang, Fall 2012 31 Adaptive HTTP Streaming System (Protocol)  Server -Can be standard web server -Media segment can be prepared in-line or off- line  Client -Sends series of HTTP GET segment requests and receives segments -Performs rate adaptation before sending a new GET segment request

32. Hui Zhang, Fall 2012 32 15-441 Fall 2011 Multimedia32 Advantages of HTTP Streaming  Easy to deploy: it's just HTTP, work with exsiting caches/proxies/CDN/Firewall  Fast startup by downloading lowest quality/smallest chunk  Bitrate switching is seamless  Many small files -Small with respect to the movie size -Large with respect to TCP 5-10 seconds of 1Mbps – 3Mbps  0.5MB – 4MB per chunk

33. Hui Zhang, Fall 2012 33 15-441 Fall 2011 Multimedia33 Example of HTTP Streaming Protocols  Apple HLS: HTTP Live Streaming  Microsoft IIS Smooth Streaming: part of Silverlight  Adobe: Flash Dynamic Streaming  DASH: Dynamic Adaptive Streaming over HTTP

34. Hui Zhang, Fall 2012 34 Terms and Definitions of Adaptive HTTP Streaming  Need -Media Presentation Description (MDP) which provides metadata For requesting (GET request) media segments For rate adaptation purpose -Segment which may include media data or metadata to decode  Use DASH an example in the few slides

35. Hui Zhang, Fall 2012 35 Example: DASH Client Thomas Stockhammer, Qualcomm, “DASH – Design Principles and Standards, Presentation at MMSys 2011

36. Hui Zhang, Fall 2012 36 Meta Data  DASH uses MPD (Media Presentation Descriptor) and Index Information as metadata for DASH Access Client  Initialization and Media Segments for Media Engine -Reuse of existing conta iner format Source: Stockhammer, Qualcomm, “DASH – Design Principles and Standards, Presentation at MMSys 2011

37. Hui Zhang, Fall 2012 37 Media Presentation Data Model MDP - description of accessible segments and corresponding timing Source: Stockhammer, Qualcomm, “DASH – Design Principles and Standards, Presentation at MMSys 2011

38. Hui Zhang, Fall 2012 38 Example of HLS Meta Data