1/23/2005 page1 11/11/2004 MPEG4 Codec for Access Grids National Center for High Performance Computing Speaker: Barz Hsu

1/23/2005 page2 Outline Definition of HDTV Brief Introduction to MPEG-4 Standard History and Problems of VIC Contributions

1/23/2005 page3 What’s High Definition Video? Frame Resolution: >= 720x480 or 640x480 Frame Rate >= 20 fps Peak Signal to Noise Ratio (PSNR) >= 30 dB

1/23/2005 page4 Another Key Factors for Access Grids Real-Time –Minimize latency & jitter Error Resilience –RTP over UDP is unreliable Large Scale Video Conferencing –Selection of suitable coding tools –Optimization Adaptive –network bandwidth –computing resources

1/23/2005 page5 Status of MPEG4 Part 2: Advanced Simple Profile (ASP) Part 10:Advanced Video Coding (AVC or H.264) Match MPEG-2 quality at up to half data rate Deliver excellent video quality across entire bandwidth spectrum – from 3G to HD –(from 40Kbps to 10MBps)

1/23/2005 page6 Performance Comparison

1/23/2005 page7 Interoperability Issues Communicate with Polycom, iChat AV (Tiger) Transport Stream –RFC 3016: RTP Payload Format for MPEG-4 Audio/Visual Streams. –RFC 3640: RTP Payload Format for Transport of MPEG-4 Elementary Streams –MPEG2TS

1/23/2005 page8 VIC: Video Conferencing Tool Developed by Network Research Group in U.C. Berkeley Mbone Tools used by Access Grid, inSORS, and VRVS –vic for video and rat for audio Based on RTP/RTCP to provide real-time video Support Multicast/Unicast Protocol Hybrid of Tcl/Tk and C++ Supported codecs –H.261, H.263, H.263++, MJPEG, nv, and etc.

1/23/2005 page9 The Problems of VIC Hard to maintain Lack of high-quality video codec –only H.261 is relative useable Limitation on frame resolution –Only QCIF (176x144), CIF (352x288) Limitation on viewing resolution Lack of exhaustive optimization on codec, render, grabber, and etc

1/23/2005 page10 Our Goal Broadcast in Television Quality –Leverage with Existing Open Source Resources VLC, MPlayer, and FFmpeg –Providing “Sense of Presence” –As Real-Time As Possible –High-Definition, High Quality –Incorporate with up-to-date codecs such as MPEG2, MPEG4 (ASP), H.264 –Less Compression Artifact –Error Resilience/Concealment

1/23/2005 page11 MPEG4 Codec Related Open Source Project –FFmpeg, xvid, and sklmp4 FFmpeg/libavcodec –High performance of CPU usage –Support various codec MPEG4, MPEG2, MPEG1 H.263++, H.263, H.261 –Provide error resilience tools slices, data partition –Cross Platform Linux, Windows, and Mac OS

1/23/2005 page12 Benefits from MPEG4 codec Support arbitrary sizes of a video stream –in contrast to H.261 –the largest size depends on your video grabber card and camera Less obvious block artifacts

1/23/2005 page13 MPEG4 Performance Typical bit-rate is 1Mbps –720x480 (NTSC) with 25 frames/sec and PSNR value is about 40 dB Consistent bit-rate –adjusting bit-rate only affects to the quality; not influence the frame- rate Up to 40 CIF, or 10 NTSC video streams.

1/23/2005 page14 Deinterlace Convert interlaced video into progressive one

1/23/2005 page15 Efficient Color Conversion VIC didn’t give correct color representation. Efficient Color Conversion –Grabber (YUYV) to Encoder (YUV420) –Decoder (YUV420) to Display (RGB15/RGB16/RGB24)

1/23/2005 page16 Scaling Viewing Windows Original VIC –only support limited sizes of viewing windows –based on unrealistic pixel replication method NCHC-VIC –scaling viewing windows arbitrarily without too much overhead –interpolation by 9 tap FIR filter –hardware assisted by Xvideo extension in XFree86 –full screen mode

1/23/2005 page17 Full Screen Snapshot

1/23/2005 page18 Real-Time Video Streaming Real-time –the compressed data are transferred at a speed that matches the coded video source rate. Latency, Jitter –average end-to-end delay –delay variation

1/23/2005 page19 Error Resilience Error Progratation

1/23/2005 page20 Error Resilience (cont’d) Resynchronization marker –the data between the error location and the first following resynchronization marker would have to be discarded. Data Partition –motion vectors and related syntactic elements are transmitted first, followed by syntactic elements like MB headers, motion vectors, and DCT coefficients.

1/23/2005 page21 Minimize Latency Multithread Encoding –Take advantage of symmetric multiple processor architecture Pipeline the latency of transmission and encoding SIMD Instruction Ses –Optimized by SSE2/SSE, MMX

1/23/2005 page22 Future Work Support H.264 codec Support MPEG-4 High Efficient AAC (HE-AAC)

1/23/2005 page23 Conclusion Project Website:

1/23/2005 page24 Thanks for your attendance