1. Image Processing and Computer Vision: 91

2. Image and Video Coding Compressing data to a smaller volume without losing (too much) information

3. Image Processing and Computer Vision: 93 Why Code Data? To reduce storage volume To reduce transmission time One colour image 760 by 580 pixels 3 channels, each 8 bits 1.3 Mbyte Video data Same resolution 25 frames per second 33 Mbyte/second

4. Image Processing and Computer Vision: 94 Redundancy Spatial Correlation between adjacent pixels Chromatic Correlation between colour channels Temporal Correlation between adjacent frames Perceptual Unnoticed losses

5. Image Processing and Computer Vision: 95 Example

6. Image Processing and Computer Vision: 96 Redundancy - Consequences Data exceeds information More data than content justifies Can lose data without losing information

7. Image Processing and Computer Vision: 97 Compression Ratio

8. Image Processing and Computer Vision: 98 Lossy vs Lossless Compression Can lose data without losing information Lossily compressed images can look similar to the original Lossy compression has greater C

9. Image Processing and Computer Vision: 99 Quality of Decoded Images Measure differences between Original Coded/decoded images Options Maximum difference Average difference Subjective scales

10. Image Processing and Computer Vision: 910 Example

11. Image Processing and Computer Vision: 911 Subjective Quality Measurement Scales - Example 0 Unusable 1 Annoying degradation 2 Adequate images 3 Barely perceptible degradation 4 No observable degradation

12. Image Processing and Computer Vision: 912 Difference coding Adjacent pixels are similar Difference is small Uncompressed: Compressed:

13. Image Processing and Computer Vision: 913 Coding Assign 4 bits to a difference Can code –7, …, +8 Overflow? Use –7 and +8 to show larger differences Code –6, …, +7 directly Use overflow codes to indicate shift of codes

14. Image Processing and Computer Vision: 914

15. Image Processing and Computer Vision: 915 Predictive Coding If signals are well sampled Adjacent samples are correlated They have similar values Differences are small Can guess next sample from value of current

16. Image Processing and Computer Vision: 916 Constants are correlation coefficient and mean grey value Difference between real and predicted values are smaller Code as for difference coding

17. Image Processing and Computer Vision: 917 Run Length Coding Replace runs of equal brightness values by (length of run, value) 1 2 2 3 3 4 4 4 5 6 5 (1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5) More use when few brightness values e.g. fax

18. Image Processing and Computer Vision: 918 Huffman Coding Uses variable length codes Most frequently occurring grey value has shortest code Least frequently occurring values have longest codes

19. Image Processing and Computer Vision: 919 Example SymbolProbability21 4 3 ABCDEFABCDEF 0.4 0.3 0.1 0.06 0.04 0.4 0.3 0.1 0.4 0.3 0.2 0.1 0.4 0.3 0.6 0.4

20. Image Processing and Computer Vision: 920 SymbolProbability21 4 3 ABCDEFABCDEF 0.4 1 0.3 00 0.1 011 0.1 0100 0.06 01010 0.04 01011 0.4 1 0.3 00 0.1 011 0.1 0100 0.1 0101 0.4 1 0.3 00 0.2 010 0.1 011 0.4 1 0.3 00 0.3 01 0.6 0 0.4 1

21. Image Processing and Computer Vision: 921 GIF Applicable to images with 256 colours Replace sequences of bytes with shorter codes Most common sequences use shortest codes

22. Image Processing and Computer Vision: 922 Wavelet coders Wavelet transform organises image content efficiently Can easily select data to be discarded

23. Image Processing and Computer Vision: 923 Wavelet coders

24. Image Processing and Computer Vision: 924 JPEG standard A subcommittee of ISO Optimised for a range of image subject matter Compression rates can be defined Quality inversely proportional to C

25. Image Processing and Computer Vision: 925 Block Transform Coding original image decompose transform quantise entropy code Sequence of 8 by 8 blocks - different planes treated separately (RGB, YUV etc.) Transformed blocks reduce redundancy and concentrate signal energy into a few coefficients discrete cosine transformation (DCT) Blocks with discarded information - goal is to smooth picture and discard information that will not be missed, e.g. high frequencies

26. Image Processing and Computer Vision: 926 Block Transform Encoding DCT Zig-zag run length code entropy code quantise 010111000111…..

27. Image Processing and Computer Vision: 927 Block Encoding 139 144 149 153 144 151 153 156 150 155 160 163 159 161 162 160 1260 -1 -12 -5 -23 -17 -6 -3 -11 -9 -2 2 -7 -2 0 1 DCT Zig-zag run length code Huffman code quantise 10011011100011…. 79 0 -1 0 -2 -1 0 0 -1 -1 0 0 0 0 0 0 79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0 0 79 1 -2 0 -1 2 -1 0 Original image

28. Image Processing and Computer Vision: 928 Block Transform Decoding DCT Zig-zag run length code entropy code quantise 010111000111…..

29. Image Processing and Computer Vision: 929 Result of Coding and Decoding 139 144 149 153 144 151 153 156 150 155 160 163 159 161 162 160 Original block 144 146 149 152 148 150 152 154 155 156 157 158 160 161 161 162 Reconstructed block -5 -2 0 1 -4 1 1 2 -5 -1 3 5 -1 0 1 -2 errors

30. Image Processing and Computer Vision: 930 Discrete Cosine Transform F[u,v] = 4C(u)C(v) n 2 n-1  j=0 n-1  k=0 f(j,k) cos (2j+1) u  2n cos (2k+1) v  2n with C(w) = 1212 For w = 0 1otherwise

31. Image Processing and Computer Vision: 931 Discussion Where are lossy steps? Quatisation and subsampling before coding How is quantisation matrix chosen? Its predefined by the standard after much experimentation

32. Image Processing and Computer Vision: 932 Video coding Many specific standards AVS, MOV, QT, … One universal standard MPEG

33. Image Processing and Computer Vision: 933 MPEG Standards Standard specifies audio, video and system layers MPEG-1: low data rates, poor quality: VHS quality at 1.5Mbits -1 MPEG-2: high quality hence high data rates: studio quality, 15Mbits -1 MPEG-4: low data rates, small images, 64 kbits -1

34. Image Processing and Computer Vision: 934 MPEG-1 Audio and video designed to work at CD ROM speeds: 1.5Mbits -1 Video 1.150Mbits -1 Audio 0.256Mbits -1 System 0.094Mbits -1

35. Image Processing and Computer Vision: 935 MPEG-2 Released in 1994 Aimed at digital TV, ATM. Additions for Interlaced video Scalable video coding Graceful degradation with noise Implementation of full standard impractical Varying profiles/levels of conformity

36. Image Processing and Computer Vision: 936 MPEG-4 Coding specifically for multimedia objects

37. Image Processing and Computer Vision: 937 Coding Algorithms Frame sequence Motion compensation Frame coding

38. Image Processing and Computer Vision: 938 Frame Sequence I frames (Intraframes) Coded independently of any other frame P frames (Predicted frames) Derived from previous I frame by motion prediction B frames (Bidirectionally interpolated) Interpolate motion compensated blocks between I and P frames

39. Image Processing and Computer Vision: 939 Motion Compensation Image is divided into macroblocks (16 x 16 pixels) Matching macroblocks are found by minimising differences Code differences and macroblock displacement

40. Image Processing and Computer Vision: 940 Frame Coding Use JPEG algorithms

41. Image Processing and Computer Vision: 941 Summary Why code data? Redundancy Assessment of compression Lossy vs. lossless compression Algorithms JPEG, MPEG

42. Image Processing and Computer Vision: 942 But what … is it good for? Engineer at the Advanced Computing Systems Division of IBM, commenting on the microchip in 1968