2. Lab #5-6 Follow-Up: More Python; Images

3. Images ● A signal (e.g. sound, temperature infrared sensor reading) is a single (one- dimensional) quantity that varies over time. ● An image (picture) can be thought of as a two-dimensional quantity that varies over space. ● Otherwise, the computations for sounds and images are about the same!

4. Black-and-White Images

7. ● Eventually we get down to a single point (pixel, “picture element”) whose color we need to represent. ● For black-and-white images, two basic options – Halftone: Black (0) or white (1); having lots of points makes a region look gray – Grayscale: Some set of values (2 N, N typically = 8) representing shades of gray.

8. ● To turn a b/w image into a grid of pixels, imagine running a stylus horizontally along the image:

9. ● This gives us a one-dimensional function showing gray level at each point along the horizontal axis:

10. ● Doing this repeatedly with lots of evenly-spaced scan lines gives us a grayscale map of the image: ● Height = grayscale value ● Spacing between lines = spacing within line

11. Color ● Light comes in difference wavelengths (frequencies):

12. ● Like a sound, energy from a given light source (e.g., star) can be described by its wavelength (frequency) spectrum : the amount of each wavelength present in the source. Spectrum of vowel “ee” Frequency (Hz) Energy (dB) Spectrum of star ICR3287 Wavelength (nm) Intensity

13. ● Light striking our eyes is focused by a lens and projected onto the retina. ● The retina contains photoreceptors (sensors) called cones that respond to different wavelengths of light: red, green, and blue. ● So retina can be thought of as a transducer that inputs light and outputs a three- dimensional value for each point in the image: [R, G, B], where R, G, and B each fall in the interval (0,1). ● So we can represent any color image as three “grayscale” images:

14. =, R, G B

15. Digital Sampling of Images ● Same questions arise for sampling images as for sound: – Sampling Frequency (how often to sample) – Quantization (# of bits per sample) ● With images, “how often” means “how many times per linear unit (inch, cm) – a.k.a. resolution (dpi) ● Focus on quantization – Each sample is either an RGB triple or an index into a color map – With too few bits, we lose gradual shading

16. Color Maps ● With 8 bits per color, 3 colors per pixel, we get 24 bits per pixel: 2 24 ≈ 17 million distinct colors ● We can actually get away with far fewer, using color maps ● Each pixel has a value that tells us what row in the color map to use ● Color map rows are R, G, B values:

17. Color Maps (in Matlab) >> colormap ans = 0 0 0.5625 0 0 0.6250... 0.4375 1.0000 0.5625 0.5000 1.0000 0.5000... 0.5625 0 0 0.5000 0 >> size(colormap) ans = 64 3

18. Quantization Problems With too few bits, we lose gradual shading: 5 bits 1 bit 2 bits 3 bits 4 bits 6 bits 7 bits 8 bits

19. Sampling and Storing Images in Files Cameras / scanners usually output images in one of several formats – JP(E)G (Joint Photographic Experts Group) – GIF (Graphics Interchange Format) – PNG (Portable Network Graphics) – TI(F)F (Tagged Image File Format) As with sound formats, main issues in image formats are compression scheme (how images are stored and transmitted to save space/time) and copyright

20. JPEG Compression Images contain lots of redundant information

21. 530 x 279 x 3 = 443610 bytes (assuming 3 bytes per pixel)

22. MOSTLY BLUE MOSTLY BROWN MOSTLY GREEN MOSTLY WHITE

23. Homebrew Compression

24. 530, 84, 151,129,235 530, 72, 255,255,255 530, 66, 0,128,64 530, 60, 128,64,0 Homebrew Compression

25. Assuming two bytes per number 4*5*2 = 40 bytes 443610 / 40 = factor of 11000! 530, 84, 151,129,235 530, 72, 255,255,255 530, 66, 0,128,64 530, 60, 128,64,0

26. Lossy Compression This sort of compression is lossy: reduces size but loses resolution JPEG offers a better lossy compromise – typically, around factor 10 without noticeable loss in quality

27. 1.Convert each RGB pixel into a form in which brightness (“Luminance”) has its own value, and use just two values (“Chrominance”, red or blue) for color. The JPEG Algorithm http://en.wikipedia.org/wiki/YCbCr

28. 2.Quantize the chroma data by a factor of two, because the eye is more sensitive to brightness than to color. The JPEG Algorithm E.g.: 8301010011 98 01100010 12301111011 20011001000 4400101100 0101000080 0110000096 01110000112 11000000192 0010000032

29. 3.Break the transformed image down into blocks of 8x8 pixels. We can represent each block as the weighted sum of various patterns, or frequency components, which will differ from block to block. The set of 64 weights becomes the new representation of the block. The JPEG Algorithm http://en.wikipedia.org/wiki/JPEG#Discrete_cosine_transform

30. 4.Quantize the frequency weights, giving more precision (bits) to the low-frequency components: again, because the eye is more sensitive to low-frequency variations (big, slowly changing patterns). JPEG Quality refers to how much we allow the high-frequency components to persist. The JPEG Algorithm

31. JPEG Quality Re-compressed with Quality = 0:

32. 5.Compress the quantized block weights using a lossless algorithm like Huffman Coding The JPEG Algorithm

33. The basic idea: Values that occur more often should be given shorter codes. In the previous sentence: Huffman Coding

34. So maybe use this code (ignoring space): Values that occur … 11110001011000100001001010010011101 …

35. Uses a tree structure to determine when we’re at the end of a code item: Huffman Code http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/Greedy/huffman.htm

36. All digital data (MS Word files, JPEG images, MP3 songs, MP4 videos) is just a sequence of numbers How we interpret those numbers depends on the program we’re using to look at the data Compressions schemes (JPEG, MP3) rely on ignoring the numbers that don’t make as much of a difference in our perception of the image, song, etc. JPEG is not a single algorithm; it’s a grab-bag of techniques that yield good results in combination Images: Summary