1. An Introduction to H.264/AVC and 3D Video Coding

2. Outline Video Coding Concepts  basic concept review  image coding structure  video coding structure H.264/AVC Introduction  history  performance comparison H.264/AVC Coding Tools  inter prediction  intra prediction  transform & quantization  de-blocking filter  entropy coding 3D Video Coding  3D video format  multiview video coding Summary with Q&A

3. Video Coding Concept -basic concept review -image coding structure -video coding structure

4. The Scope of Image and Video Coding Standardization Only the Syntax and Decoder are standardized:

5. Images and Video

6. Needs for Video Compression Without compression  Visual telephony (e.g. CIF @ 15 frames/s): 325 (pels) x 288 (lines) x 15 (farmes/s) x 1.5 bytes = 18.25 Mbit/s  Digital TV (ITU-T 601 4:2:0 @30 frames/s): 720 (pels) x 480 (lines) x 30 (farmes/s) x 1.5 bytes = 124.4 Mbit/s  HDTV (e.g. 1280x720 pels 4:2:0 @ 60 frames/s): Compression results in lower bit rates  Lower transmission and storage cost

7. RGB vs. YCbCr [1/3]

8. RGB vs. YCbCr [2/3]

9. RGB vs. YCbCr [3/3]

10. Common YCbCr Formats

11. Subjective View

12. Block Based Coding [1/2]

13. Block Based Coding [2/2]

14. Group of Picture (GOP)

15. Video Coding Concept -basic concept review -image coding structure -video coding structure

16. Image Coding Structure

17. Transform

18. Quantization S:0 1 2 3 4 5 6 7 (3 bits) Quantization: Quantization step-size Q=2: S/2 Quantization Levels (Q):0 0 1 1 2 2 3 3 (2 bits) Inverse quantization (x2):0 0 2 2 4 4 6 6 Quantization error:0 1 0 1 0 1 0 1 Quantization step-size Q=4: S/4 Quantization Levels (Q):0 0 0 0 1 1 1 1 (2 bits) Inverse quantization (x4):0 0 0 0 4 4 4 4 Quantization error:0 1 2 3 0 1 2 3

19. Effect of DCT + Quantization

20. Entropy coding

21. Video Coding Concept -basic concept review -image coding structure -video coding structure

22. Temporal Redundancy [1/2] The amount of data to be coded can be reduced significantly

23. Standard Video Encoder

24. Block Based Motion Compensation [1/2]

25. Algorithms for Motion Estimation Full Search  Guarantee find the global minimum SAD  high computational complexity Fast Search  Local minimum SAD  Low computational complexity  Reduce candidate blocks  Reduce matching pixels in candidate blocks

26. Diamond Search

27. Video coding structure

28. H.264/AVC Introduction -History -Performance comparison

29. History

30. Joint Video Team

31. MPEG-2 Has Hit A Wall

32. MPEG-4 in Comparison

33. H.26L Provides Focus

34. MPEG-4 “Adopts” H.264

35. State of the Art Standards MPEG-2  DVD, DVT, since 1994 MPEG-4  DVR, Digital Still Camera, since 1999  ~1.5x coding gain over MPEG-2 (ASP) MPEG-4 part 10, AVC (H.264)  Mobile video, DVB-H, Blu-ray Disc and etc.  2~3x coding gain over MPEG-2

36. AVC Profiles

37. coding tools and profiles

38. H.264/AVC Introduction -History -Performance comparison

39. Compare to Other Standard Fair comparisons of H.26L(TML-8.0) versus H.263v3,MPEG-2,and MPEG-4  TML-8.0 at half of the bit rate as MPEG-4 for the same visual fidelity  Source from VCEG-N18.doc (Soptember,2001) Objective evaluation  Average improvement of TML-8.0over MPEG-2 (VM-5) of 5.8 dB PSNR (peak gain 7.2 dB) for equal bandwidths  TML-8.0 average gain of 3.1 dB relative to H.263++ (High-Latency Profile) for equal bandwidths (up to 5.2 dB)  Gain of 2.2 dB over MPEG-4 (Advanced Simple Profile) for equal bandwidths (max. 3.6 dB)

40. Test Sets “Streaming” Test:  Four QCIF sequences coded at 10 Hz and 15 Hz (Foreman, Container, News, Tempete)  Four CIF sequences coded at 15 Hz and 30 Hz (Bus, Flower, Garden, Mobile and Calendar, and Tempete)  With B frame “Real-Time Conversation” Test:  Four QCIF sequences encoded at 10Hz and 15Hz (Akiyo, Foreman, Mother and Daughter, and Silent Voice)  Four CIF sequences encoded at 15Hz and 30Hz (Carphone, Foreman, Paris, and Sean)  Without B frames

41. Objective evaluation [1/2]

42. Objective evaluation [2/2]

43. Subjective evaluation Example: Sequence Mobile, frame 40

44. Perceptual Test of H.264/AVC High Profile

45. Objective Performance of H.264/AVC High Profile

46. Intra mode performance [1/2]  Average gain of H.264 to JPEG: 5.2 dB (luma)  Average gain of H.264 to JPEG2000: 1.12 dB (luma)  Average gain of Motion JPEG2000 to H.264: 1.42 dB (chroma)  The smaller the bit rate, the higher the gain of H.264

47. Intra mode performance [2/2]

48. Intra mode performance [chroma]

49. Intra mode performance [FRExt] a set of 8 photographic monochrome test images  with resolutions from 512x512 up to 2048x3072 samples Average gain of H.264/AVC HP to JPEG2000: 0.5 dB  over the entire test image set and all bit-rates

50. JPEG2000 vs. H.264 Intra

51. H.264/AVC Coding Tools -Inter prediction -Intra prediction -Transform and Quantization -De-blocking Filter -Entropy Coding

52. Basic Coding Structure

53. Standard Tools Comparison

54. Motion Compensation

55. Macro Block Partitions

56. Example – Frame 1

57. Example – Frame 2

58. Example – Residual [no MC]

59. Example – Residual [16x16]

60. Example – Residual [8x8]

61. Example – Residual [4x4]

62. Example of Variable Block Sizes Large block means  Less bits for MVs  More bits on residuals Small block means  More bits for MVs  Less bits on residuals

63. Summary Key Features  Enhances motion compensation  Small blocks for transform coding  De-blocking filter 50% bit rate saving against any other standards, by  Better prediction  More computation  More memory Video coding layer is still based on hybrid video coding algorithm, buy with important differences