NUS.SOC.CS5248 Ooi Wei Tsang Systems Support for Continuous Media OOI WEI TSANG

NUS.SOC.CS5248 OOI Wei Tsang Scope of this class Systems Support: Application Middleware Operating System Continuous Media: Video, audio, animation

NUS.SOC.CS5248 OOI Wei Tsang Local Media Applications Watch Movie Listen to Music Video Editing : (Much Less Interesting …)

NUS.SOC.CS5248 OOI Wei Tsang Network Media Applications Live Webcast Pre-recorded Webcast Video Conferencing Video on Demand

NUS.SOC.CS5248 OOI Wei Tsang Categories Interactive Non- Interactive Live Video Conferencing NUSLive, NUSCast Pre- recorded Lecture/Video on Demand

NUS.SOC.CS5248 OOI Wei Tsang Streaming Media Requires Reliable networks Low latency Bounded latency Plenty of bandwidth Not in today’s Internet.

NUS.SOC.CS5248 OOI Wei Tsang Overview Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS5248 OOI Wei Tsang We will NOT Design new codec Study media processing CS6212/CS5240 Attempt to improve the Internet EE5910

NUS.SOC.CS5248 Ooi Wei Tsang Summary of Topics Covered

NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Review of Media Compression Network Communication Model Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Adaptation Video on Demand Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Buffering Audio/Video Synchronization Packet Loss Recovery Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS5248 OOI Wei Tsang Sample of Topics Covered Caching Application-Level Multicast Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS5248 OOI Wei Tsang Lecture Philosophy Cover the breadth, rather than depth Classic papers, rather than recent papers

NUS.SOC.CS5248 Ooi Wei Tsang Class Organization

NUS.SOC.CS5248 OOI Wei Tsang Goals Learn about existing techniques and systems Learn about current research efforts Learn about how to do research

NUS.SOC.CS5248 OOI Wei Tsang Philosophy Research oriented Students are expected to be mature and independent What you learned is more important than grade

NUS.SOC.CS5248 OOI Wei Tsang Please don’t ask.. “Is this equation important?” “Is this equation examinable?” “Do I have to memorize this header format?” “Post answer to quiz/survey/review”

NUS.SOC.CS5248 OOI Wei Tsang Please do ask.. “What is the effect of changing k in the equation?” “Why did the designer added this bit in the header?”

NUS.SOC.CS5248 OOI Wei Tsang Academic Honesty No copying among students No copying from published work ZERO TOLERENCE to Plagiarism

NUS.SOC.CS5248 OOI Wei Tsang Discussion? Strongly encouraged.. but must acknowledge all contributions write up solutions independently

NUS.SOC.CS5248 OOI Wei Tsang Gilligan’s Island Rule Discuss with friends Throw notes away Go watch “Gilligan’s Island” (or “Phua Chu Kang”, or “Super Sunday”) Write down your solution

NUS.SOC.CS5248 Ooi Wei Tsang Continuous Assessments

NUS.SOC.CS5248 OOI Wei Tsang Continuous Assessments Project (50%) Paper Survey (20%) Paper Review (15%) One of Programming Assignment Random Quizzes Class Participation Mid-term

NUS.SOC.CS5248 OOI Wei Tsang Research Project Define a problem Propose a solution Implement the solution Evaluate the solution

NUS.SOC.CS5248 OOI Wei Tsang Types of Project New problem + solution Existing problem + new solution Existing problem + existing solutions + comparisons + new insights See website for more..

NUS.SOC.CS5248 OOI Wei Tsang Variations Combine with your own research subjected to your advisor’s approval Combine with another class project subjected to the other instructor’s approval must be more significant

NUS.SOC.CS5248 OOI Wei Tsang Research Project Deliverables: page paper poster presentation demo (if applicable) In a group of 1 to n students

NUS.SOC.CS5248 OOI Wei Tsang Survey Paper In-depth study of a topic 4-10 papers expected Suggested list of topics available (can be the same topic as your project!)

NUS.SOC.CS5248 OOI Wei Tsang Survey Paper summarize recent results organize, generalize, categorize add understanding

NUS.SOC.CS5248 OOI Wei Tsang Survey Paper Deliverables: page survey paper Individual work

NUS.SOC.CS5248 OOI Wei Tsang Paper Reviews Read, summarize, and criticize 10 papers One is assigned almost weekly Deliverables: One page report per paper

NUS.SOC.CS5248 OOI Wei Tsang Expected Workload Read 3-4 papers per week Some hours of coding per week

NUS.SOC.CS5248 OOI Wei Tsang Office Hours Saturday 11 am – 12 noon SOC or for appointment or just pop in

NUS.SOC.CS5248 OOI Wei Tsang Web Sites Main Website IVLE Website

NUS.SOC.CS5248 Ooi Wei Tsang Questions

NUS.SOC.CS5248 Ooi Wei Tsang Review of Media Compression

NUS.SOC.CS5248 OOI Wei Tsang You are Here Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS5248 OOI Wei Tsang Why compress? “Bandwidth Not Enough” “Disk Space Not Enough” Size of Uncompressed DVD Movie =

NUS.SOC.CS5248 OOI Wei Tsang Why compress? “Bandwidth Not Enough” “Disk Space Not Enough” Size of Uncompressed DVD Movie = 720 x 576 x 3 x 25 x 60 x 120 = 208.6GB

NUS.SOC.CS5248 Ooi Wei Tsang JPEG Compression

Original Image (1153KB) 1:1

Original Image (1153KB) 3.5:1

Original Image (1153KB) 17:1

Original Image (1153KB) 27:1

Original Image (1153KB) 72:1

Original Image (1153KB) 192:1

NUS.SOC.CS5248 OOI Wei Tsang Compression Ratio QualitySizeRatio Raw TIFF1153KB1:1 Zipped TIFF982KB1.2:1 Q=100331KB3.5:1 Q=7067KB17:1 Q=4043KB27:1 Q=1016KB72:1 Q=16KB192:1

NUS.SOC.CS5248 OOI Wei Tsang Magic of JPEG Throw away information we cannot see Color information “Low-frequency signals” Rearrange data for good compression Use standard compression

NUS.SOC.CS5248 OOI Wei Tsang Throw away color information Y VU

NUS.SOC.CS5248 OOI Wei Tsang Color Sub-sampling 4:4:4 4:2:2 4:2:0 4:1:1

Original Image (1153KB)

NUS.SOC.CS5248 OOI Wei Tsang Discrete Cosine Transform DCT demo applet and source code is available on the web.

NUS.SOC.CS5248 OOI Wei Tsang Quantization /=

NUS.SOC.CS5248 OOI Wei Tsang Differential Coding

NUS.SOC.CS5248 OOI Wei Tsang Zig-zag ordering , 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0

NUS.SOC.CS5248 OOI Wei Tsang Run-Length Encoding , 3, 2, 4, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0 (27, 1) (3, 1) (2, 1), (4, 1), (1, 2), (0, 5), (1, 1), (0, 4)

NUS.SOC.CS5248 OOI Wei Tsang Idea: Motion JPEG Compress every frame in a video as JPEG DVD-quality video = 208.6GB Reduction ratio = 27:1 Final size = 7.7GB

NUS.SOC.CS5248 Ooi Wei Tsang Video Compression

NUS.SOC.CS5248 OOI Wei Tsang Temporal Redundancy

NUS.SOC.CS5248 OOI Wei Tsang Motion Estimation

NUS.SOC.CS5248 OOI Wei Tsang Bi-directional Prediction

NUS.SOC.CS5248 OOI Wei Tsang H.261 I-Frame P-Frame

NUS.SOC.CS5248 OOI Wei Tsang MPEG-1 B-Frame

NUS.SOC.CS5248 OOI Wei Tsang MPEG Frame Pattern Example display sequence: IBBPBBP … Example encoding sequence: IPBBPBB

NUS.SOC.CS5248 OOI Wei Tsang Compression Ratio Frame TypeTypical Ratio I10:1 P20:1 B50:1

NUS.SOC.CS5248 OOI Wei Tsang Sequence sequence header: width height frame rate bit rate :

NUS.SOC.CS5248 OOI Wei Tsang GOP: Group of Picture gop header: time :

NUS.SOC.CS5248 OOI Wei Tsang Picture pic header: number type (I,P,B) :

NUS.SOC.CS5248 OOI Wei Tsang Picture

NUS.SOC.CS5248 OOI Wei Tsang Slice

NUS.SOC.CS5248 OOI Wei Tsang Macroblock

NUS.SOC.CS5248 OOI Wei Tsang Block YY YY U V 1 Macroblock =

NUS.SOC.CS5248 OOI Wei Tsang MPEG in daily life MPEG Standards Bit-rateUsage MPEG-11.5MbpsVCD MPEG Mbps DVD, SVCD, HDTV MPEG-4Scalable QuickTime, DivX;)

NUS.SOC.CS5248 OOI Wei Tsang Impact on Systems Design How to package data into packets? How to deal with packet loss? How to deal with bursty traffic? How to predict decoding time? :