1. Block-based coding Multimedia Systems and Standards S2 IF Telkom University

2. Why block-based video coding? the choice of a suitable video coder depends on the associated application and available resources Primary reasons: The quality of service they are designed to achieve achieve fairly high compression ratios in real-time scenarios. Exp: H.263  < 64 kbit/s, 25 f/s and an average PSNR of 30 dB the scalability of their output bit rates source code is available 2

3. Video frame format based on the Common Intermediate Format standardize the horizontal and vertical resolutions in pixels of YCbCr sequences 3

4. Chrom = ½ Lum the human eye is less sensitive to the details of the colour information 4

5. Layering structure video frame = k x 16 lines of pixels k = 1 for sub-QCIF, 9 for QCIF, 18 for CIF, 4CIF and 16CIF Video frames = Group of Blocks (GOB) 6 for sub-QCIF, 9 for QCIF and 18 for CIF, 4CIF and 16CIF GOB = macroblocks (MB) MB = 4 luminance blocks + 2 spatially corresponding colour difference blocks block = 8 pixels by 8 lines of Y, U or V 5

6. QCIF: picture - GOB 6

7. QCIF: MB - block 7

8. Methods video frame is divided into a number macroblocks (16x16 blocks of pixels) two coding modes: INTRA coded as an independent still image without any reference to precedent frames  spatial redundancies INTER exploits the temporal redundancies between successive frames higher compression efficiency by employing predictive coding 8

9. INTER & INTRA coding coding only the difference between a frame and its reference  INTER frame or predicted frame (P- frame) coding if successive frames are not strongly correlated  INTRA frame (I-Frame) coding in INTER frame coding, some MBs could still be INTRA coded  mode flag in each MB 9

10. INTER mode motion search: similarities between the current frame and the reference one the difference image (the residual error frame) is DCT-transformed and quantized converted to a one-dimensional matrix of coefficients using the zigzag-pattern coding to exploit the long runs of zeros that appear in the picture after quantization. run-length coder (Huffman coder) assigns variable-length codes to the non-zero levels and the runs of zeros in the resulting one-dimensional matrix. 10

11. INTER vs quality an accumulation of INTER coded frames could lead to fuzzy picture quality the effect of repeated quantization Insert INTRA frame: to refresh the picture quality after a certain number of INTER frames 11

12. Motion estimation INTER coding: the block matching (BM) motion estimation process Each MB in the current frame is compared to MBs of the previous reconstructed frame within a search window of user-defined size 12

13. The matching criterion only luminance is used may use any error measure e.g. mean square error (MSE), sum of absolute difference (SAD), MAD, SSE, SATD, etc. motion vector (MV): displacement vector between the current MB and its best match vertical and horizontal components 13

14. 14

15. INTRA within P-frame If all SADs corresponding to 16x16 matrices within the search window < a certain motion activity threshold then the current MB is INTRA coded within the P-frame 15

16. ITU-T H.263 16

17. Operation For each MB: the SADs are compared to a motion activity threshold to decide whether INTRA or INTER mode is to be used for a specific MB INTRA mode: the coefficients of the six 8x8 blocks are DCT transformed, quantized, zigzag coded, run-length coded, and then variable-length coded using a Huffman encoder 17

18. INTER mode: the resulting MV is differentially coded and the same encoding procedure as in INTRA mode is applied on the residual matrix Operation (cont’d) 18

19. 2-D 8x8 DCT 19

20. Example of DCT DC coefficients: assigned an 8-bit length codeword (coded more accurately) AC coefficients: run-length coded 20

21. Quantization controls the coding efficiency & the quality of the reconstructed video sequence based on the human visual sensitivity Techniques: scalar quantization: each sample is quantized independently vector quantization: quantization of a group of samples or vectors 21

22. Quantization: lossy maps the values of the DCT transformed coefficients to a smaller range of values the exact original pixel value cannot be restored 22

23. H.263 Quantization Quantisation INTRADC coefficient: LEVEL = COF/8 INTRA AC coefficients: LEVEL = |COF|/(2 x Qp) INTER coefficients: LEVEL = (|COF| - Qp/2)/(2 x Qp) Inverse quantisation INTRADC coefficient: COF’ = LEVEL x 8 INTRA or INTER coefficients: |COF’| = 0if LEVEL=0 |COF’| = 2Qp x LEVEL + Qp if LEVEL<> 0, Qp is odd |COF’| = 2Qp x LEVEL x Qp -1if LEVEL <> 0, Qp is even 23

24. example 24

25. zigzag scan Result? 25

26. Some block-based DCT video coding standards Standar d ApplicationBit rates MPEG-1Audio/video storage on CD- ROM 1,5 – 2 Mbps MPEG-2HDTV/DVB4 – 9 Mbps H.261Video over ISDNp x 64 kbps H.263Video over PSTN< 64 kbps 26