Block-based coding Multimedia Systems and Standards S2 IF Telkom University

Why block-based video coding? the choice of a suitable video coder depends on the associated application and available resources Primary reasons: The quality of service they are designed to achieve achieve fairly high compression ratios in real-time scenarios. Exp: H.263  < 64 kbit/s, 25 f/s and an average PSNR of 30 dB the scalability of their output bit rates source code is available 2

Video frame format based on the Common Intermediate Format standardize the horizontal and vertical resolutions in pixels of YCbCr sequences 3

Chrom = ½ Lum the human eye is less sensitive to the details of the colour information 4

Layering structure video frame = k x 16 lines of pixels k = 1 for sub-QCIF, 9 for QCIF, 18 for CIF, 4CIF and 16CIF Video frames = Group of Blocks (GOB) 6 for sub-QCIF, 9 for QCIF and 18 for CIF, 4CIF and 16CIF GOB = macroblocks (MB) MB = 4 luminance blocks + 2 spatially corresponding colour difference blocks block = 8 pixels by 8 lines of Y, U or V 5

QCIF: picture - GOB 6

QCIF: MB - block 7

Methods video frame is divided into a number macroblocks (16x16 blocks of pixels) two coding modes: INTRA coded as an independent still image without any reference to precedent frames  spatial redundancies INTER exploits the temporal redundancies between successive frames higher compression efficiency by employing predictive coding 8

INTER & INTRA coding coding only the difference between a frame and its reference  INTER frame or predicted frame (P- frame) coding if successive frames are not strongly correlated  INTRA frame (I-Frame) coding in INTER frame coding, some MBs could still be INTRA coded  mode flag in each MB 9

INTER mode motion search: similarities between the current frame and the reference one the difference image (the residual error frame) is DCT-transformed and quantized converted to a one-dimensional matrix of coefficients using the zigzag-pattern coding to exploit the long runs of zeros that appear in the picture after quantization. run-length coder (Huffman coder) assigns variable-length codes to the non-zero levels and the runs of zeros in the resulting one-dimensional matrix. 10

INTER vs quality an accumulation of INTER coded frames could lead to fuzzy picture quality the effect of repeated quantization Insert INTRA frame: to refresh the picture quality after a certain number of INTER frames 11

Motion estimation INTER coding: the block matching (BM) motion estimation process Each MB in the current frame is compared to MBs of the previous reconstructed frame within a search window of user-defined size 12

The matching criterion only luminance is used may use any error measure e.g. mean square error (MSE), sum of absolute difference (SAD), MAD, SSE, SATD, etc. motion vector (MV): displacement vector between the current MB and its best match vertical and horizontal components 13

14

INTRA within P-frame If all SADs corresponding to 16x16 matrices within the search window < a certain motion activity threshold then the current MB is INTRA coded within the P-frame 15

ITU-T H

Operation For each MB: the SADs are compared to a motion activity threshold to decide whether INTRA or INTER mode is to be used for a specific MB INTRA mode: the coefficients of the six 8x8 blocks are DCT transformed, quantized, zigzag coded, run-length coded, and then variable-length coded using a Huffman encoder 17

INTER mode: the resulting MV is differentially coded and the same encoding procedure as in INTRA mode is applied on the residual matrix Operation (cont’d) 18

2-D 8x8 DCT 19

Example of DCT DC coefficients: assigned an 8-bit length codeword (coded more accurately) AC coefficients: run-length coded 20

Quantization controls the coding efficiency & the quality of the reconstructed video sequence based on the human visual sensitivity Techniques: scalar quantization: each sample is quantized independently vector quantization: quantization of a group of samples or vectors 21

Quantization: lossy maps the values of the DCT transformed coefficients to a smaller range of values the exact original pixel value cannot be restored 22

H.263 Quantization Quantisation INTRADC coefficient: LEVEL = COF/8 INTRA AC coefficients: LEVEL = |COF|/(2 x Qp) INTER coefficients: LEVEL = (|COF| - Qp/2)/(2 x Qp) Inverse quantisation INTRADC coefficient: COF’ = LEVEL x 8 INTRA or INTER coefficients: |COF’| = 0if LEVEL=0 |COF’| = 2Qp x LEVEL + Qp if LEVEL<> 0, Qp is odd |COF’| = 2Qp x LEVEL x Qp -1if LEVEL <> 0, Qp is even 23

example 24

zigzag scan Result? 25

Some block-based DCT video coding standards Standar d ApplicationBit rates MPEG-1Audio/video storage on CD- ROM 1,5 – 2 Mbps MPEG-2HDTV/DVB4 – 9 Mbps H.261Video over ISDNp x 64 kbps H.263Video over PSTN< 64 kbps 26