1. Chapter 5 Bitmapped Images Multimedia Systems

2. Key Points For displayed images, physical dimension = pixel dimension/device resolution. For displayed images, physical dimension = pixel dimension/device resolution. To display an image at its natural size it is scaled by device resolution/image resolution. To display an image at its natural size it is scaled by device resolution/image resolution. Downsampled images may look better than images stored at device resolution. Downsampled images may look better than images stored at device resolution.

3. Key Points Lossless compression: no information is lost during a compression/decompression cycle. Lossy compression: some information is discarded during the compression process. Lossless compression: no information is lost during a compression/decompression cycle. Lossy compression: some information is discarded during the compression process. Run-length encoding (RLE), Huffman encoding, and LZ77, LZ78 and LZW are lossless methods. GIFs use LZW. Run-length encoding (RLE), Huffman encoding, and LZ77, LZ78 and LZW are lossless methods. GIFs use LZW. JPEG compression is based on the use of the Discrete Cosine Transform (DCT). It is the most important lossy method for images. JPEG compression is based on the use of the Discrete Cosine Transform (DCT). It is the most important lossy method for images.

4. Key Points GIFs are best for simple shapes with areas of flat colour. JPEGs are best for photographs and other images with continuous tones. GIFs are best for simple shapes with areas of flat colour. JPEGs are best for photographs and other images with continuous tones. Selecting an area of pixels defines a mask which protects the rest of the image from changes that are applied. Selecting an area of pixels defines a mask which protects the rest of the image from changes that are applied. A grey-scale mask is called an alpha channel and can be used for feathering and compositing, amongst other things. A grey-scale mask is called an alpha channel and can be used for feathering and compositing, amongst other things. Pixel point processing: for a pixel with value p, we compute a new value p' = f(p). Brightness and contrast adjustments are common examples of pixel point processing. Pixel point processing: for a pixel with value p, we compute a new value p' = f(p). Brightness and contrast adjustments are common examples of pixel point processing.

5. Key Points Pixel group processing is based on a convolution process that also takes account of the values of neighboring pixels. Filters such as blurring and sharpening are the typical applications of pixel group processing. Pixel group processing is based on a convolution process that also takes account of the values of neighboring pixels. Filters such as blurring and sharpening are the typical applications of pixel group processing. In general, applying geometrical transformations to bitmapped images requires interpolation of pixels and may cause a loss of image quality. In general, applying geometrical transformations to bitmapped images requires interpolation of pixels and may cause a loss of image quality.

6. Resolution A measure of how finely a device approximates continuous images using finite pixels A measure of how finely a device approximates continuous images using finite pixels Dots per inch (dpi) Dots per inch (dpi) Printer: 600 dpi Printer: 600 dpi imagesetter (book production): 1200-2700 dpi imagesetter (book production): 1200-2700 dpi Scanner: 300-3600 dpi Scanner: 300-3600 dpi Drum scanners, transparency scanners: higher resolution Drum scanners, transparency scanners: higher resolution Lines per inch Lines per inch Colour printer of 150 lines per inch Colour printer of 150 lines per inch As much as 5 times lower than number of DPI As much as 5 times lower than number of DPI Frame size Frame size NTSC: 640 by 480 NTSC: 640 by 480 PAL: 768 by 576 PAL: 768 by 576 Computer monitors Computer monitors VGA: 640 by 480 VGA: 640 by 480 1027 by 768 1027 by 768 Dots per inch, 72 dpi Dots per inch, 72 dpi

7. Resolution Physical dimension Physical dimension physical dimension = pixel dimension / device resolution where device resolution is measured in pixels per unit length where device resolution is measured in pixels per unit length Image has pixel dimensions(ex. 300 by 200 pixels), but no physical dimension Image has pixel dimensions(ex. 300 by 200 pixels), but no physical dimension To display image at its natural size, record its pixels per inch (ppi) To display image at its natural size, record its pixels per inch (ppi)

8. Resolution If image's resolution < display device's resolution then interpolation of device's pixels (scaled up) If image's resolution < display device's resolution then interpolation of device's pixels (scaled up) If image's resolution > display device's resolution then discard image pixels (downsampling) If image's resolution > display device's resolution then discard image pixels (downsampling) Oversampling Oversampling Sampling images at a higher resolution than its ultimately displayed Sampling images at a higher resolution than its ultimately displayed Web browsers are poor at down-sampling Web browsers are poor at down-sampling down-sampling in advance down-sampling in advance

9. Image Compression Original data Compressed data encodedecode Lossless compression No information is lost Run-length encoding  Huffman coding  LZ77, LZ78, LZW,… Original data Compressed data Decompressed data encodedecode Lossy compression Discard some information JPEG

10. Lossless Compression Run-length encoding (RLE) Run-length encoding (RLE) replacing a run of consecutive pixels of the same color by color value and its count replacing a run of consecutive pixels of the same color by color value and its count ex. 00001100 0 4 ; 1 2 ; 0 2 ex. 00001100 0 4 ; 1 2 ; 0 2 encode color valuecount

11. Lossless Compression Huffman coding Huffman coding Using variable-length codes (VLC) according to its occurrence times Using variable-length codes (VLC) according to its occurrence times The more frequently it appears, the less code length it requires, and vice versa The more frequently it appears, the less code length it requires, and vice versa Dictionary-based compression Dictionary-based compression Construction a dictionary of strings, all occurrences of a string are replaced by a pointer into the dictionary Construction a dictionary of strings, all occurrences of a string are replaced by a pointer into the dictionary LZ77, LZ78, LZW (a variation of LZ78), … LZ77, LZ78, LZW (a variation of LZ78), … LZW: patented by Unisys LZW: patented by Unisys GIF: LZW, PNG: LZ77, PKZIP: LZ77 GIF: LZW, PNG: LZ77, PKZIP: LZ77

12. JPEG Compression JPEG JPEG Develop by Joint Photographic Experts Group Develop by Joint Photographic Experts Group Collaboration between ISO and CCITT (now ITU) Collaboration between ISO and CCITT (now ITU) Suit to compress digit photographs and bitmapped images Suit to compress digit photographs and bitmapped images Most widely known standard for lossy image compression Most widely known standard for lossy image compression

13. JPEG Encoding Step Shift pixel value (0~255 => -128~127) Shift pixel value (0~255 => -128~127) Divide image into blocks of size 8  8 Divide image into blocks of size 8  8 Use Discrete Cosine Transform (DCT) for each blocks Use Discrete Cosine Transform (DCT) for each blocks Quantize coefficients in block Quantize coefficients in block Apply Huffman codes for quantized coefficient Apply Huffman codes for quantized coefficient

14. Discrete Cosine Transform DCT of an N  N pixel image DCT of an N  N pixel image  DCT uv = Where 0 <= u < N, 0 <= v < N, 1 / 2 for u, v = 0 1 otherwise p xy is the pixel value of x, y coordinate C u, C v =

15. Quantization Discard high frequencies information which do not contribute much perceived quality Discard high frequencies information which do not contribute much perceived quality 16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87 80 62 18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 72 92 95 98 112 100 103 99 Sample quantization table 39.8 6.5 -2.4 1.2 -0.3 -1.0 0.7 1.1 -102.4 4.5 2.2 1.1 0.3 -0.6 -1.0 -0.4 37.7 1.3 1.7 0.2 -1.5 -2.2 -0.1 0.2 -5.6 2.2 -1.3 -0.8 1.4 0.2 0.1 0.1 -3.3 -0.7 -1.7 0.7 -0.6 -2.6 -1.3 0.7 5.9 -0.1 -0.4 -0.7 1.9 -0.2 1.4 0.0 3.9 5.5 2.3 -0.5 0.0 -0.8 -0.5 -0.1 -3.4 0.5 -1.0 0.8 0.9 0.0 0.3 0.0 DCT coefficients in frequency domain 2 1 0 0 0 0 0 0 -9 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 After quantization low high Quantize

16. JPEG Encoding After quantization After quantization Many components end up with zero coefficients Many components end up with zero coefficients Fewer bits are needed to store non-zero coefficients Fewer bits are needed to store non-zero coefficients Zeros are run-length encoded Zeros are run-length encoded Scan with zig-zag sequence to max the length of runs of zeroes Scan with zig-zag sequence to max the length of runs of zeroes Apply Huffman coding Apply Huffman coding

17. JPEG Encoding Example 124 125 122 120 122 119 117 118 121 121 120 119 119 120 120 118 126 124 123 122 121 121 120 120 124 124 125 125 126 125 124 124 127 127 128 129 130 128 127 125 143 142 143 142 140 139 139 139 150 148 152 152 152 152 150 151 156 159 158 155 158 158 157 156 An 8 * 8 block Shift & DCT transform 39.8 6.5 -2.4 1.2 -0.3 -1.0 0.7 1.1 -102.4 4.5 2.2 1.1 0.3 -0.6 -1.0 -0.4 37.7 1.3 1.7 0.2 -1.5 -2.2 -0.1 0.2 -5.6 2.2 -1.3 -0.8 1.4 0.2 0.1 0.1 -3.3 -0.7 -1.7 0.7 -0.6 -2.6 -1.3 0.7 5.9 -0.1 -0.4 -0.7 1.9 -0.2 1.4 0.0 3.9 5.5 2.3 -0.5 0.0 -0.8 -0.5 -0.1 -3.4 0.5 -1.0 0.8 0.9 0.0 0.3 0.0 DCT coefficients in frequency domain Quantization 2 1 0 0 0 0 0 0 -9 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 After using quantization table on coefficients 2 1 0 0 0 0 0 0 -9 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Zig-zag scan Follow the scan to code coefficients

18. JPEG Decoding How to Decompression ? Just reverse compression process sequence! (but there is some information lost!) 32 11 0 0 0 0 0 0 -108 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39.8 6.5 -2.4 1.2 -0.3 -1.0 0.7 1.1 -102.4 4.5 2.2 1.1 0.3 -0.6 -1.0 -0.4 37.7 1.3 1.7 0.2 -1.5 -2.2 -0.1 0.2 -5.6 2.2 -1.3 -0.8 1.4 0.2 0.1 0.1 -3.3 -0.7 -1.7 0.7 -0.6 -2.6 -1.3 0.7 5.9 -0.1 -0.4 -0.7 1.9 -0.2 1.4 0.0 3.9 5.5 2.3 -0.5 0.0 -0.8 -0.5 -0.1 -3.4 0.5 -1.0 0.8 0.9 0.0 0.3 0.0 Frequency domain after decompression Original frequency domain

19. Compression Artefact Original image After JPEG compression (Produced by Photoshop)

20. Image Manipulation Why manipulate image? Why manipulate image? Correct deficiencies caused by creation or digitization Correct deficiencies caused by creation or digitization ex. Red-eye removal ex. Red-eye removal Create unnatural image Create unnatural image ex. Creating a glow around an object ex. Creating a glow around an object Image manipulation program Image manipulation program Adobe Photoshop, PhotoImpact Adobe Photoshop, PhotoImpact

21. Selections, Masks and Alpha Channels Selection Selection Defining an area within the image Defining an area within the image Tools Tools rectangular or elliptical marquee, lasso, rectangular or elliptical marquee, lasso, magic wand, and magnetic lasso Photoshop tools Elliptical marquee lasso magic wand

22. Selections, Masks and Alpha Channels Mask Mask The unselected area protect from any changes you make to the image The unselected area protect from any changes you make to the image Use single bit for each pixel: ‘1’ for all masked out pixels and ‘0’ for selection Use single bit for each pixel: ‘1’ for all masked out pixels and ‘0’ for selection Alpha channel Alpha channel Using more than one bit to specify transparency Using more than one bit to specify transparency compute composited image of a pixel p compute composited image of a pixel p where p 1, p 2 : the values of pixels in two original images α : normalized to lie between 0 and 1 α : normalized to lie between 0 and 1 p = αp 1 + (1 – α)p 2

23. Pixel Point Processing Compute a pixel’s new value p’ on its old value p, without regard to other pixel Compute a pixel’s new value p’ on its old value p, without regard to other pixel where f: mapping function ex. Construction a negative from greyscale(0~255) image f(p) = 255 - p f(p) = 255 - p p’ = f(p)

24. Pixel Point Processing Colour correction and alteration Colour correction and alteration Brightness Brightness Adjust the value of each pixel up or down uniformly Adjust the value of each pixel up or down uniformly Increasing makes every pixel lighter, decreasing make darker Increasing makes every pixel lighter, decreasing make darker Contrast Contrast Adjust the range between lightest and darkest areas of the image Adjust the range between lightest and darkest areas of the image Increasing contrast makes light areas lighter and dark areas darker, and vice versa Increasing contrast makes light areas lighter and dark areas darker, and vice versa Fig. 5.9 S-curve for enhancing contrast Fig. 5.9 S-curve for enhancing contrast

25. Pixel Group Processing Computing each pixel’s new value as a function of old value and neighboring pixels Computing each pixel’s new value as a function of old value and neighboring pixels Original Image Convolution mask New image Convolution kernel * =  ( ) i

26. Pixel Group Processing Original image Convolution mask New image a b c d h ef gi p (x,y) p’ = ap x-1,y+1 + bp x,y+1 + cp x+1,y+1 + dp x-1,y + ep x,y + fp x+1,y + gp x-1,y-1 + hp x,y-1 + ip x+1,y-1 p’

27. Filter Blur Blur Filter Filter ex. ex. In signal processing, smoothing effect is obtained by filtering out high frequencies (low pass filter) In signal processing, smoothing effect is obtained by filtering out high frequencies (low pass filter) 1/9 1/9 1/9 117 117 27 27 1/9 1/9 1/9 * 117 117 27 27 117 105 57 27 117 117 27 27

28. Filter Gaussian blur Gaussian blur Gaussian bell curve Gaussian bell curve Roughly 70% contribution to the centre pixel Roughly 70% contribution to the centre pixel Sharpen Sharpen Filter Filter ex. ex. -1 -1 -1 -1 9 -1 -1 -1 -1 117 117 51 27 27 117 317 -45 -45 27 117 117 51 27 27 -1 -1 -1 -1 9 -1 -1 -1 -1 *

29. Filter Unsharp masking Unsharp masking Accentuate the edges, but retain the other parts Accentuate the edges, but retain the other parts Idea Idea Blurring filters out high frequencies Blurring filters out high frequencies If remove blurred image away from origin, it left high frequencies – the sharp edges If remove blurred image away from origin, it left high frequencies – the sharp edges Step Step Copy original image, applying a Gaussian blur to it Copy original image, applying a Gaussian blur to it Subtracting the blurred mask pixel value from original pixel value with scaling factor Subtracting the blurred mask pixel value from original pixel value with scaling factor

30. origin blur sharp Unsharp mask (Produced by Photoshop)

31. Filter Image I blur Image I’Image I’’ sharp Image I blur Image I’Image I’’ sharp Is I = I’’ ?

32. Geometrical Transformations Scaling, translation, reflection, rotation and shearing Scaling, translation, reflection, rotation and shearing Two transformation schemes Two transformation schemes Forward mapping Forward mapping Assign original pixel value to transformed pixel Assign original pixel value to transformed pixel Inverse mapping Inverse mapping For each transformed pixel, find its inverse pixel value in original image For each transformed pixel, find its inverse pixel value in original image Advantage Advantage No missing pixels No missing pixels Disadvantage Disadvantage Require inverse mapping Require inverse mapping

33. Sampling Methods of Inverse Mapping P1P1 P2P2 P3P3 P4P4 If transformed pixel ‘X’ locates among P 1, P 2, P 3 and P 4  Nearest neighbor method: I(‘X’) = I(P 3 ) where I(p) is the intensity value of pixel p  Bi-linear interpolation: I(‘X’) = (1-a)(1-b)I( P 1 ) + a(1-b)I(P 2 ) + (1-a)bI(P 3 ) + abI(P 4 ) where a, b are the fractional parts of ‘X’  Bi-cubic interpolation: based on cubic splines

34. Original image * 2 Bi-linear interpolation Bi-cubic interpolation