0. JVT Coding ITU-T H.26L; ISO MPEG-4, Part 10
Thomas Wiegand
Heinrich Hertz Institute, Berlin, Germany
Associated Rapporteur ITU-T VCEG
Co-Chair ITU-T/ISO JVT

1. Outline JVT/H.26L: History, Goals, Applications, Structure
Video Coding Layer
Syntax and Decoder
Coder Control
Comparisons
Network Adaptation Layer

2. The JVT Project
New ITU-T Q.6/SG16 (VCEG - Video Coding Experts Group) standardization activity for video compression
August 1999: 1st test model (TML-1)
December 2001: Formation of the Joint Video Team (JVT) between VCEG and MPEG to finalize H.26L as a joint project: JVT Coding (similar to MPEG-2/H.262)
February 2002: WD-2 (11 th test model: TML-11)
Schedule:
February 2002: Last major feature adoptions
November 2002: Final approval

3. Goals of the JVT/H.26L Project
Simple syntax specification
Targeting simple and clean solutions
Avoiding any excessive quantity of optional features or profile configurations
Improved Coding Efficiency
Average bit rate reduction of 50% given fixed fidelity compared to any other standard
Improved Network Friendliness
Issues examined in H.263 and MPEG-4 are further improved
Major targets are mobile networks and Internet

4. Applications Conversational H.32X Services H.320 Conversational
3GPP Conversational H.324/M
3GPP Conversational IP/RTP/SIP
H.323 Conversational Internet/unmanaged/best effort IP/RTP
Streaming Services
3GPP Streaming IP/RTP/RTSP
Streaming IP/RTP/RTSP
Other Services
Entertainment Satellite/Cable/DVD, 0.5 – 8 Mbit/s
Digital Cinema Application
3GPP Multimedia Messaging Services

5. JVT/H.26L Layer Structure
Control Data
Video Coding Layer
Macroblock
Data Partitioning
Slice/Partition
Network Adaptation Layer
H.320
H.324
H.323/IP
H.324M
etc.

6. JVT/H.26L Layer Structure
Control Data
Video Coding Layer
Macroblock
Data Partitioning
Slice/Partition
Network Adaptation Layer
H.320
H.324
H.323/IP
H.324M
etc.

7. H.26L Video Coding Layer - Coder Control Control Data
Entropy
Coding
Transform/ Quantizer
Quant. Transf. coeffs -
Decoder
Deq./Inv. Transform
Motion-
Compensated
Predictor
Intra/Inter
Motion
Data
Motion
Estimator

8. Common Elements with other Standards
16x16 macroblocks
Conventional sampling of chrominance and association of luminance and chrominance data
Block motion displacement
Motion vectors over picture boundaries
Variable block-size motion
Block transforms (not wavelets or fractals)
Run-length coding of transform coefficients
Scalar quantization
I- and P-picture types

9. Motion Compensation Accuracy
Coder
Control
Control
Data
Entropy
Coding
Transform/ Quantizer
Quant. Transf. coeffs -
Decoder
Deq./Inv. Transform
16x16 1
8x16 MB
Modes
8x8 2 3
16x8
Motion-
Compensated
Predictor
Intra/Inter
8x8
4x8 1 2 3
4x4
8x4
Modes
Motion
Data
Motion
Estimator
1/4 (QCIF) or 1/8 (CIF) pel

10. Tree-Structured MB Partition
INTRA
MB:
INTRA
8x8:
Allows motion segmentation shapes like
INTRA

11. Multiple Reference Frames
Coder
Control
Control
Data
Entropy
Coding
Transform/ Quantizer
Quant. Transf. coeffs -
Decoder
Deq./Inv. Transform
Motion-
Compensated
Predictor
Multiple Reference Frames for
Motion Compensation
Intra/Inter
Motion
Data
Motion
Estimator

12. Motion Compensation
Various block sizes and shapes for motion compensation (7 segmentations of the macroblock: 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4)
1/4 sample (sort of per MPEG-4) and 1/8 sample accuracy motion
6x6 tap filtering to 1/2 sample accuracy, bilinear filtering to 1/4 sample accuracy, special position with heavier filtering
8x8 tap filtering applied repeatedly for 1/8 pel motion
Multiple reference pictures (per H Annex U)
Temporally-reversed motion and generalized B-frames
B-frame prediction weighting

13. Residual Coding - Residual coding is based on 4x4 blocks
Integer Transform
Coder
Control
Control
Data
Entropy
Coding
Transform/ Quantizer
Quant. Transf. coeffs -
Decoder
Deq./Inv. Transform
Motion-
Compensated
Predictor
Intra/Inter
Motion
Data
Motion
Estimator

14. Residual and Intra Coding
Transform
Based primarily on 4x4 transform (all prior standards: 8x8)
Expanded to 8x8 for chroma by 2x2 transform of the DC values
Intra Coding Structure
Directional spatial prediction (6 types luma, 1 chroma)
Expanded to 16x16 for luma intra by 4x4 transform of the DC values

15. Quantization and Deblocking
Two inverse scan patterns
Logarithmic step size control
Smaller step size for chroma (per H.263 Annex T)
Deblocking Filter (in loop)

16. Entropy Coding - Coder Control Data Transform/ Quantizer
Deq./Inv. Transform
Motion-
Compensated
Predictor
Control
Data
Quant. Transf. coeffs
Motion
Intra/Inter
Coder
Decoder
Estimator
Transform/ Quantizer -
Entropy
Coding

17. Exp-Golomb Coding One table that is used universally for all symbols
Simple, but has the following disadvantages
Probability distribution may not be a good fit
Probability distribution is static
Correlations between symbols are ignored, i.e. no conditional probabilities are used
Code words must have integer number of bits (Low coding efficiency for highly peaked pdfs)

18. Context-based Adaptive Binary Arithmetic Codes (CABAC)
Usage of adaptive probability models
Exploiting symbol correlations by using contexts
Non-integer number of bits per symbol by using arithmetic codes
Restriction to binary arithmetic coding
Simple and fast adaptation mechanism
Fast binary arithmetic coders are available
Binarization is done using the UVLC

19. SP-Pictures
SP-pictures are much smaller than I-pictures

20. Comparison of JVT/H.26L and MPEG-4
Both:
Sequence structure IBBPBBP...
Search range: 32x32 around 16x16 predictor
Encoders use similar D+lR optimization techniques
MPEG-4: Advanced Simple Profile (ASP)
Motion Compensation: 1/4 pel
Global Motion Compensation
QPB=1.2 x QPP
H.26L:
Motion Compensation: 1/4 pel (QCIF), 1/8 pel (CIF)
Using CABAC entropy coding
5 reference frames
QPB=QPP+2

21. Comparison to MPEG-2, H.263, MPEG-4
Foreman QCIF 10Hz 39 38 37
Left-hand side
Right-hand side 36
JVT/H.26L 35
MPEG-4 34
MPEG-2
Quality
Y-PSNR [dB] 33
H.263 32 31 30 29 28 27 50
100
150
200
250
Bit-rate [kbit/s]

22. Comparison to MPEG-2, H.263, MPEG-4
Tempete CIF 30Hz 38
???-hand side 37 36 35 34 33
Quality
Y-PSNR [dB] 32 31 30 29
JVT/H.26L 28
MPEG-4 27
MPEG-2 26
H.263 25
500
1000
1500
2000
2500
3000
3500
Bit-rate [kbit/s]

23. Comparison to MPEG-2, H.263, MPEG-4
Tempete CIF 30Hz 38
Left-hand side
Right-hand side 37 36 35 34 33
Quality
Y-PSNR [dB] 32 31 30 29
JVT/H.26L 28
MPEG-4 27
MPEG-2 26
H.263 25
500
1000
1500
2000
2500
3000
3500
Bit-rate [kbit/s]

24. JVT/H.26L Layer Structure
Control Data
Video Coding Layer
Macroblock
Data Partitioning
Slice/Partition
Network Adaptation Layer
H.320
H.324
H.323/IP
H.324M
etc.

25. Network Adaptation Layer
Tasks
Mapping of slice structure on transport layer
Setup, framing, encapsulation, interleaving, logical channels, closing, timing issues, synchronization, etc.
Transport of control and header information
Further network specific issues (feedback, prioritization,…)
The specification for each NAL includes
Verbal description
Encapsulation process (processing of slice structure)
Header and parameter set specification

26. Network Adaptation Features
Slice Structure Coding
Slices for a specified number of macroblocks
Slices for a specified number of Bytes
Data Partitioning: header, motion vectors, Intra, and Inter transform coefficients
Signalling of header info and parameter sets via appropriate means

27. Conclusions
JVT/H.26L contains a video coding and a network adaptation layer
Video coding layer is based on hybrid video coding and similar in spirit to other standards but with important differences
Bit-rate savings up to 50 % against any other standard
Network adaptation layer and error resilience features consider transport over MPEG2/H.222.0
IP and 3GPP networks
Download of documents and software via anonymous ftp to standard.pictel.com/video-site