CDEEP Lecture Video Adaptation to mobile devices Ganesh Narayana Murthy Guided by: Prof. Sridhar Iyer

Problem Definition Adapt CDEEP videos to be viewable on mobile devices: – Viewable at low network bandwidths (like GPRS) – Viewable at low cost Video bit-rate – Size of video stream over time – Total size = bit-rate * total time – CDEEP video bit-rate: 1150kbps – GPRS bit-rate: 40kbps The problem: – Video playing incurs delays if available network bandwidth is less than video-bit rate

Video Transcoding Converting from one video format to another – Changing video bit rate – Changing other parameters like frame rate, screen resolution Format NameTypical Bit RateApplication MPEG-11.5Mbps or lessCD-ROM MPEG-25-8MbpsDVD, HDTV H.263Typically low bit rates Low bit-rate video conferencing MPEG-4 / H.26440Kbps to 10Mbps and above Internet Streaming, Video Telephony Flash Video (FLV)Typically low bit rates Embedded video in websites

Video Quality at low-bit rates (a) MPEG-1(b) MPEG-2 Images from transcoded videos (Target bit rate : 40kbps, No audio)

Video Quality at low bit-rates (contd.) (c) H.264 (mp4) (d) H.263 (3gpp) Images from transcoded videos (Target bit rate : 40kbps, No audio)

(e) Flash Video (flv) Images from transcoded videos (Target bit rate : 40kbps, No audio)

Comparison of Video Codecs Format Name Original Video Size Converted Video Size Video Quality at low-bit rates Remarks MPEG-1432MB26MBPoorCannot be used at low bit-rates MPEG-2432MB29.12MBPoorCannot be used at low bit rates H MB38.3MBPoorCannot be used at low bit rates H MB16.9MBGoodProcessing power / Decoding complexity is high.[1] Flash432MB20.5MBGoodCan be used, but cost is still high. (Note: Video bit rate = 1150kbps, No audio, Target bit rate = 40kbps, No audio) Video Sizes are still high for viewing over GPRS

Study-Element Based Adaptation

Motivation CDEEP video usually consists of – Presentation slides – Instructor explaining on white paper – Video of instructor talking Presentation slide is usually not changing – Video of slide is not required. One image is sufficient Idea – Extract one image every ‘n’ seconds and send to client. – This would reduce amount of data sent for showing one slide.

Method-1 Extract one image every ‘n’ seconds – Server sends one image every ‘n’ seconds to client – Audio is simultaneously streamed Network bandwidth and Size – Network Overhead (NO) = Image Size / n – Size Overhead (SO) = Total size of images What is the user experience?

User Experience Basis Presentation Study Element – Portion of video showing one slide White Paper Study Element – Portion of video showing instructor writing on white paper Instructor Study Element – Portion of video showing instructor talking ………… Presentation Slide Delay in start of slide 3 ……… White Paper Video Time (secs)

User Experience Presentation Element – Delay Experienced (D2) = Delay in start of slide as compared to audio White Paper Element – Delay Experienced (D1) = Delay between any two consequitive images = Sending Rate Instructor Element – Only audio important. No image need be sent. So, no delay or user experienced considered. User Experience (U i ) = 1 sec / D i

Method-2 Trade-off for user experience Cost incurred in terms of number of images sent Same sending rate for all elements, cannot balance user experience and cost. Choose different sending rate for each study element Probably: – Lower sending rate for white paper element – Higher sending rate for presentation element User Experience Cost Sending Rate Trade-Off Relation

System Overview

Building the index Corpus of 10 videos – Representative of various departments Consider different sending rates ‘r’ – For each ‘r’ find NO,SO and U for every study element in a video. – Repeat for all videos and take average. This relation can be used backwards: – For calculating sending rate, given network bandwidth and user experience.

Graphs of User Experience Presentation ElementWhite Paper Element

Graphs of overheads White Paper ElementPresentation Element

Results Original Video Size(MB) Images Size (MB) Reduction (%) U1U2Supported Network Bandwidth kbps and above Achieved Size Reduction Fig: Video stream size reduction (note: Original video bit-rate = 1150kbps, No audio)

Results (contd.) White Paper PresentationU1U2NO1 (kbps) S01 (MB) NO2 (kbps) SO2 (MB) Total Size (SO) Different Sending Rates Required Network Bandwidth =max(NO1,NO2) is reduced Reduction in size user experience for white paper element remaining same

Conclusion Large size reduction can be achieved by using the concept of slideshows Identifiying study-elements within the video helps define user-experience of the slideshow. CDEEP Lecture videos can be adapted to low network bandwidths and in a cost-controlled manner.

Future Work 1.Automated tagging – Identifying study element boundaries using shot detection techniques 2.System Implementation – Building a full system implementing the idea – Study the actual working.

References 1. H.264 white paper. 2.Real-time Content-Based Adaptive Streaming of Sports Videos. Shih-Fu Chang, Di Zhong, and Raj Kumar. In CBAIVL '01: Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), page 139, Washington, DC, USA, IEEE Computer Society. 3.Content-aware video adaptation under low-bitrate constraint. Ming-Ho Hsiao, Yi-Wen Chen, Hua-Tsung Chen, Kuan-Hung Chou, and Suh-Yin Lee. EURASIP J. Adv. Signal Process,2007(2):27-27, A Characteristics-Based Bandwidth Reduction Technique for Pre-recorded Videos. Wallapak Tavanapong and Srikanth Krishnamohan. In IEEE International Conference on Multimedia and Expo (III), pages , 2000.

Questions?

Content-Aware Adaptation Method NameAdaption MechanismVideo QualityRemarks Hsiao et.al.[2]Identify visual attention regions in a frame. Encode them at high quality. PoorQuality of important objects still depends on network bandwidth Chang.et.al [3]Identify events in sports videos at high quality. Other regions as slideshows. GoodSlideshow of images reduces network bandwidth and size Tavanapong. et. Al. [4] Identify non-changing portions of lecture video and extract one image from them GoodExploits redundancy in lecture videos