1. Overview of Error Resiliency Schemes in H.264/AVC Standard Sunil Kumar, Liyang Xu, Mrinal K. Mandal, and Sethuraman Panchanathan Elsevier Journal of Visual Communication and Image Representation, to appear in2005. S. Wenger, “ H.264/AVC over IP, ” IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645 – 656, July 2003

2. Outline Introduction Error-Resilience Tools in H.264/AVC Experimental Results References

3. Introduction H.264/AVC Video Coding Standard –http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/20 0310132110/Overview_of_the_H.264_AVC_Video_Codin g_Standard.ppthttp://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/20 0310132110/Overview_of_the_H.264_AVC_Video_Codin g_Standard.ppt

4. Introduction Error-resilience tools in older video compression standard –Picture segmentation (slices, GOBs); –Placement of Intra MBs, Intra slices, and intra pictures; –Reference picture selection (with and without feedback); –Data partitioning MPEG 1 part 2 MPEG 2 part 2 H.261 H.263

5. Error-Resilience Tools in H.264/AVC Reference picture selection –Feedback-based –Without feedback e.g. Video redundancy coding Picture segmentation –Slice Flexible macroblock ordering (FMO) –7 modes –Overhead bits highly depends on the picture format, the content, and the QP. (S. Wenger and M. Horowitz) < 5% penalty at QP = 16; on average 20% at QP = 28. EncoderDecoder feedback channel I0 p1p3p5p7 p9 p2p4p6p8 Channel 1 Channel 2 Checker board mode Interleaving mode

6. Error-Resilience Tools in H.264/AVC SP/SI frame http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200402181718/The_SP-_and_SI- Frames_Design_for_H.264_AVC.ppt http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200402181718/The_SP-_and_SI- Frames_Design_for_H.264_AVC.ppt –Instantaneous refreshing –Gradual refreshing P 1, n-2 P 1, n-1 S 1, n P 1, n+1 S 21, n SI 1, n P 1, n-3

7. Error-Resilience Tools in H.264/AVC Intra placement –Used to combat drifting effects. –ConstrainedIntraPrediction flag An Intra block can not be predicted from an Inter block. –IDR slice and IDR picture An IDR picture contains only IDR slices. An IDR slice can only be part of an IDR picture. An IDR picture invalidates all short-term reference memory buffers. Intra inter Intra I P IDR P

8. Error-Resilience Tools in H.264/AVC Data partitioning –For unequal error protection (UEP). –Three different partition types in H.264. –Type A Header information (MB types, QP, MVs) –Type B Intra coded block patterns (CBPs) and Intra coefficients –Type C Inter CBPs and Inter coefficients –Importance A > B > C

9. Error-Resilience Tools in H.264/AVC Redundant slices (RSs) –For a MB, an encoder can place redundant representation of the same MBs into the same it stream. –e.g. One slice is coded using different QP. If the slice of low QP is available, the decoder discards the RS; otherwise, the RS is reconstructed by the decoder Slice A QP1 Slice A QP2 Decoder

10. Error-Resilience Tools in H.264/AVC Parameter sets –Including picture size, entropy coding method, MV resolution, and so on. –Sequence parameter set Containing all information related to the picture sequence between two IDR pictures. –Picture parameter set Containing all information related to all slices in a picture. –e.g. Sending multiple copies of SPSs so to enhance the arrival rate. –e.g. SPSs can be sent out-of-band.

11. Error-Resilience Tools in H.264/AVC Intra block refreshing by RD control – –Better than periodically inserting intra MBs.

12. Experimental Results Stephan Wenger

13. Experimental Results Error concealment scheme –Intra –Inter

14. foreman paris Stephan Wenger

15. Experimental Results Experiment 1: One picture, one packet, without any error-resilience –Packet loss rates > 3% unprotected H.264 video becomes unusable. Foreman Paris Stephan Wenger

16. Experimental Results Experiment 2: One picture, one packet, with intra macroblock refresh –Very good PSNR results –Very high amount of Intra MBs of significant size A coarser quantizer is needed for CBR – less spatial detail. Stephan Wenger Foreman PSNR = 35 PLR = 0%

17. Experimental Results Experiment 3: 2-4 slices in one frame –Unfavorable PSNR results for Foreman. –2 nd best PSNR results for Paris. Foreman Paris Packet header overhead ? Stephan Wenger

18. Experimental Results Experiment 4: Slices interleaving –Better than Exp. 3 for Foreman. –Worse than Exp. 3 for Paris. Foreman Paris Stephan Wenger

19. Experimental Results Experiment 5: Data partitioning –Partition A is sent twice (3% PLR) or 3 times (>3% PLR). Degraded PSNR due to CBR. Foreman Paris Stephan Wenger

20. Experimental Results Experiment 6: FMO –Randomly inserting Intra MB. (3% of PLR) –Best PSNR results Foreman Paris Stephan Wenger

21. Experimental Results QCIF, 10fps,100Kbps M. M. Ghandi and M. Ghanbari

22. Experimental Results QCIF, 10fps,100Kbps M. M. Ghandi and M. Ghanbari

23. References S. Kumar, L. Xu, M. K. Mandal, and S. Panchanathan, “ Overview of Error Resiliency Schemes in H.264/AVC Standard, ” Elsevier Journal of Visual Communication and Image Representation (Special issue on H.264/AVC Video Coding Standard), to appear in 2005. S. Wenger, “ H.264/AVC over IP, ” IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645 – 656, July 2003. S. Wenger and M. Horowitz, “ Scattered slices: A new error resilience tool for H.26L, ” in JVT-B027, Feb 2002. M. M. Ghandi and M. Ghanbari, “ Layered H.264 video transmission with hierarchical QAM, ” Elsevier J. of Visual Communication and Image Representation (Special issue of H.264/AVC), to appear in 2005.