Image Processing and Computer Vision: 91

Image and Video Coding Compressing data to a smaller volume without losing (too much) information

Image Processing and Computer Vision: 93 Why Code Data? To reduce storage volume To reduce transmission time One colour image 760 by 580 pixels 3 channels, each 8 bits 1.3 Mbyte Video data Same resolution 25 frames per second 33 Mbyte/second

Image Processing and Computer Vision: 94 Redundancy Spatial Correlation between adjacent pixels Chromatic Correlation between colour channels Temporal Correlation between adjacent frames Perceptual Unnoticed losses

Image Processing and Computer Vision: 95 Example

Image Processing and Computer Vision: 96 Redundancy - Consequences Data exceeds information More data than content justifies Can lose data without losing information

Image Processing and Computer Vision: 97 Compression Ratio

Image Processing and Computer Vision: 98 Lossy vs Lossless Compression Can lose data without losing information Lossily compressed images can look similar to the original Lossy compression has greater C

Image Processing and Computer Vision: 99 Quality of Decoded Images Measure differences between Original Coded/decoded images Options Maximum difference Average difference Subjective scales

Image Processing and Computer Vision: 910 Example

Image Processing and Computer Vision: 911 Subjective Quality Measurement Scales - Example 0 Unusable 1 Annoying degradation 2 Adequate images 3 Barely perceptible degradation 4 No observable degradation

Image Processing and Computer Vision: 912 Difference coding Adjacent pixels are similar Difference is small Uncompressed: Compressed:

Image Processing and Computer Vision: 913 Coding Assign 4 bits to a difference Can code –7, …, +8 Overflow? Use –7 and +8 to show larger differences Code –6, …, +7 directly Use overflow codes to indicate shift of codes

Image Processing and Computer Vision: 914

Image Processing and Computer Vision: 915 Predictive Coding If signals are well sampled Adjacent samples are correlated They have similar values Differences are small Can guess next sample from value of current

Image Processing and Computer Vision: 916 Constants are correlation coefficient and mean grey value Difference between real and predicted values are smaller Code as for difference coding

Image Processing and Computer Vision: 917 Run Length Coding Replace runs of equal brightness values by (length of run, value) (1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5) More use when few brightness values e.g. fax

Image Processing and Computer Vision: 918 Huffman Coding Uses variable length codes Most frequently occurring grey value has shortest code Least frequently occurring values have longest codes

Image Processing and Computer Vision: 919 Example SymbolProbability ABCDEFABCDEF

Image Processing and Computer Vision: 920 SymbolProbability ABCDEFABCDEF

Image Processing and Computer Vision: 921 GIF Applicable to images with 256 colours Replace sequences of bytes with shorter codes Most common sequences use shortest codes

Image Processing and Computer Vision: 922 Wavelet coders Wavelet transform organises image content efficiently Can easily select data to be discarded

Image Processing and Computer Vision: 923 Wavelet coders

Image Processing and Computer Vision: 924 JPEG standard A subcommittee of ISO Optimised for a range of image subject matter Compression rates can be defined Quality inversely proportional to C

Image Processing and Computer Vision: 925 Block Transform Coding original image decompose transform quantise entropy code Sequence of 8 by 8 blocks - different planes treated separately (RGB, YUV etc.) Transformed blocks reduce redundancy and concentrate signal energy into a few coefficients discrete cosine transformation (DCT) Blocks with discarded information - goal is to smooth picture and discard information that will not be missed, e.g. high frequencies

Image Processing and Computer Vision: 926 Block Transform Encoding DCT Zig-zag run length code entropy code quantise …..

Image Processing and Computer Vision: 927 Block Encoding DCT Zig-zag run length code Huffman code quantise … Original image

Image Processing and Computer Vision: 928 Block Transform Decoding DCT Zig-zag run length code entropy code quantise …..

Image Processing and Computer Vision: 929 Result of Coding and Decoding Original block Reconstructed block errors

Image Processing and Computer Vision: 930 Discrete Cosine Transform F[u,v] = 4C(u)C(v) n 2 n-1  j=0 n-1  k=0 f(j,k) cos (2j+1) u  2n cos (2k+1) v  2n with C(w) = 1212 For w = 0 1otherwise

Image Processing and Computer Vision: 931 Discussion Where are lossy steps? Quatisation and subsampling before coding How is quantisation matrix chosen? Its predefined by the standard after much experimentation

Image Processing and Computer Vision: 932 Video coding Many specific standards AVS, MOV, QT, … One universal standard MPEG

Image Processing and Computer Vision: 933 MPEG Standards Standard specifies audio, video and system layers MPEG-1: low data rates, poor quality: VHS quality at 1.5Mbits -1 MPEG-2: high quality hence high data rates: studio quality, 15Mbits -1 MPEG-4: low data rates, small images, 64 kbits -1

Image Processing and Computer Vision: 934 MPEG-1 Audio and video designed to work at CD ROM speeds: 1.5Mbits -1 Video 1.150Mbits -1 Audio 0.256Mbits -1 System 0.094Mbits -1

Image Processing and Computer Vision: 935 MPEG-2 Released in 1994 Aimed at digital TV, ATM. Additions for Interlaced video Scalable video coding Graceful degradation with noise Implementation of full standard impractical Varying profiles/levels of conformity

Image Processing and Computer Vision: 936 MPEG-4 Coding specifically for multimedia objects

Image Processing and Computer Vision: 937 Coding Algorithms Frame sequence Motion compensation Frame coding

Image Processing and Computer Vision: 938 Frame Sequence I frames (Intraframes) Coded independently of any other frame P frames (Predicted frames) Derived from previous I frame by motion prediction B frames (Bidirectionally interpolated) Interpolate motion compensated blocks between I and P frames

Image Processing and Computer Vision: 939 Motion Compensation Image is divided into macroblocks (16 x 16 pixels) Matching macroblocks are found by minimising differences Code differences and macroblock displacement

Image Processing and Computer Vision: 940 Frame Coding Use JPEG algorithms

Image Processing and Computer Vision: 941 Summary Why code data? Redundancy Assessment of compression Lossy vs. lossless compression Algorithms JPEG, MPEG

Image Processing and Computer Vision: 942 But what … is it good for? Engineer at the Advanced Computing Systems Division of IBM, commenting on the microchip in 1968