Representing Images

Goals for Image Representation digitization & resolution digitization & resolution representing color representing color color depth (bits per pixel) color depth (bits per pixel) image compression (lossy vs lossless) image compression (lossy vs lossless)

Picture is represented by “dots”, or pixels (picture elements). Other examples: newsprint newsprint television television computer monitor. computer monitor. More dots = higher quality = higher quality = higher resolution = higher resolution (ppi = pixels/inch)

Digitization examples Place grid over image; color square if more than half is black.

Digitization examples Place grid over image; color square if more than half is black. etc..

Typical resolutions… 225 x 225 pixels (small image) 854 x 480 and higher 1920 x 1080 and ~ 100 ppi (i.e., 100x100) 300dpi + details… 12 megapixels

Space considerations Why not use highest resolution (best quality) Why not use highest resolution (best quality) Tradeoff between quality and space (file size on disk; time to transmit over network) Tradeoff between quality and space (file size on disk; time to transmit over network) 100 x 100 = 10,000 bits, but 100 x 100 = 10,000 bits, but 200 x 200 = 40,000 bits, and 200 x 200 = 40,000 bits, and 300 x 300 = 90,000 bits 300 x 300 = 90,000 bits SPACE GROWS QUADRATICALLY: n x n = n 2 SPACE GROWS QUADRATICALLY: n x n = n 2

Color Representation What kind of images can we create if we use 1 bit per pixel? How can we get more colors?

Color Representation IDEA: represent a pixel with 3 bits: 000 = no dot lit (black) 111 = R, G, B lit (white light) 100 = R lit (pure red) 110 = R, G lit (yellow light) Each pixel has 3 color dots: RGB: the three primary colors of light. PROBLEM: only allows 8 different pixel colors

3-bit color (8 colors) dithering Dithering: to create the illusion of different colors in a region, use different colored pixels. E.g., orange = alternate red and yellow pixels

Color Depth (also “Bit Depth”) Allocate more bits for each of R, G, B to describe amount of light (intensity) for each color. 6-bit color depth: = intensity 1 for red intensity 3 for green intensity 0 for blue Allows for ______________ different colors what is code for pink? 2 6 = 64

“True color” = 24 bits = 8 bits/color millions of colors: more than eye can discern Indicate RGB by 3 (decimal) numbers between 0 and 255 [example: 102, 223, 46 ] OR by 6 hex values = 2 per color = 8 bits/color) [example: 66DF2E] Color Depth

Color Cube R, G, B are three axes of a three- dimensional “cube” of color

Color Depth & Quality 1 bit4 bits 8 bits24 bits

Space considerations Quality depends on: resolution, color depth. Space required depends on: resolution, color depth. Bits Needed = num pixels x bits/pixel e.g. 600x600, truecolor --> x 24 = 8,640,000 bits = 1,080,000 bytes = 1.08 Mb file (.bmp)

More on Resolutions…. er-resolution.html er-resolution.html er-resolution.html er-resolution.html –resolutions greater than 300dpi refer to individual color dots used to make up a single pixel. al-versus-film-resolution.html al-versus-film-resolution.html al-versus-film-resolution.html al-versus-film-resolution.html –“grain size” for film comparable to resolution on high- end digital camera (about 5 microns). Resolution.html Resolution.html Resolution.html Resolution.html –sdtv versus hdtv (various versions)

Compression Run-length encoding of image: lossless.gif works by reducing 24 bit color by rounding to 256 colors (8 bits total per pixel) derived from the image, then using LZ compression. Good for drawings, images without great color depth..jpg (too complex to describe here) advantages over gif: many colors (note: uses Huffman coding) Pros and cons: gif good for sharp lines, bad for good color. jpg good for photographs..gif and.jpg are lossy compression techniques.

Vector-based representations (as opposed to “bit-map” =.bmp,.gif,.jpg) Store equations of lines, edges, regions Store equations of lines, edges, regions Very efficient for geometric contents Very efficient for geometric contents Scales beautifully Scales beautifully Scalable fonts without jaggies Scalable fonts without jaggies Flash files Flash files Disadv: hard to obtain directly from scanned images, hard to edit pixel-wise Disadv: hard to obtain directly from scanned images, hard to edit pixel-wise

Video Video = many, many, many, many images Video = many, many, many, many images Typical 30 frames per second. Typical 30 frames per second. Space requirement is simply: Space requirement is simply: (space per image) x (#seconds) x 30 Example: 600 x 600 truecolor image = 1.08 Mb Thus, roughly 30Mb per second seconds in a 90 min video 5400 seconds in a 90 min video 5400 x 30Mb = Mb = 162Gb 5400 x 30Mb = Mb = 162Gb (without compression) DVD capacity 4.7Gb. Blue Ray 50 Gb Special temporal compression techniques used for Video

Summary Higher Resolution (pixels/inch) gives better quality, but space increases quadratically. Higher Resolution (pixels/inch) gives better quality, but space increases quadratically. Bit depth (color depth) measured in bits/pixel. “Truecolor” = 24 bits, 8 for each of R, G, B. Bit depth (color depth) measured in bits/pixel. “Truecolor” = 24 bits, 8 for each of R, G, B. Space = num_pixels x bits/pixel Space = num_pixels x bits/pixel Compression techniques: run-length,.gif,.jpg. Compression techniques: run-length,.gif,.jpg. Vector-based representations have adv, disadv. Vector-based representations have adv, disadv. Video is big; compression is both spatial and temporal Video is big; compression is both spatial and temporal See for a good survey. See for a good survey.

Representing Sounds

Goals Understand how sound is represented digitally Understand how sound is represented digitally Understand sampling rate, quantization and tradeoff between quality and file size Understand sampling rate, quantization and tradeoff between quality and file size Know typical factors used in sound compression Know typical factors used in sound compression

Sound Representation Sound is a (continuous) wave To represent digitally, decide on: Sampling rate Sampling rate –how often to read the amplitude of the wave Quantization Quantization –how many bits used to represent the amplitude These two values are analogous to resolution and color depth for images…

Sampling sampling rates vs. quality of sound sampling rates vs. quality of sound slow rate misses peaks, valleys time intensity of compression wave (e.g., as measured by voltage from microphone)

Sampling - examples Not a very good approximation of the original sound curve

Sampling - examples How much sampling is enough? A better approximation of the original sound curve Flash tutorial

Quantization quantization vs. quality of sound quantization vs. quality of sound quantization Number of distinct amplitudes we choose to represent. Fewer values available creates large rounding errors

Quantization - examples Number of distinct amplitudes we can represent depends on the number of bits we use. Values must be rounded to nearest number… in the 1-bit case, “0” or “1” 1 0

Quantization - examples With 2 bits, we can represent 4 values (0,.33,.66, 1). We still must round, but rounding error is less, and curve is more closely approximated.

A/D and D/A converters Sound Cards and what they do:

Space considerations Sample Rate x Quantization x Duration Sample Rate x Quantization x Duration Raw audio 44.1Khz x 16 bits = 705Kbits/sec =.7 Mbits/sec stereo = 1.4 Mbits/sec 3-minute song = 252 Mbits ≈ 32 Mbytes 3-minute song = 252 Mbits ≈ 32 Mbytes 16 Gb iPod holds 16000/32 = 500 songs (without compression. with compression: 4000 songs) 16 Gb iPod holds 16000/32 = 500 songs (without compression. with compression: 4000 songs)

Compression Use limitations of human hearing to decide on sampling rate and quantization. Use limitations of human hearing to decide on sampling rate and quantization. Human ear can’t hear much better than 20Khz, so sampling at 44.1Khz is adequate. Human ear can’t hear much better than 20Khz, so sampling at 44.1Khz is adequate. Other limitations of our hearing, as well as Other limitations of our hearing, as well as psychoacoustics, are used in compression algorithms (e.g. masking effects), together with other compression techniques (e.g. Huffman coding). psychoacoustics Typical file types:.wav (raw),.mp3 (compressed) Typical file types:.wav (raw),.mp3 (compressed)

Summary Sound is analog, we have digital representation Sound is analog, we have digital representation D/A and A/D converters used on sound cards D/A and A/D converters used on sound cards Sampling rate is how many samples/sec are represented. (Typical = 44.1K) Sampling rate is how many samples/sec are represented. (Typical = 44.1K) Quantization is how many distinct amplitude values we can represent. (Typical = 16 bits: 65536) Quantization is how many distinct amplitude values we can represent. (Typical = 16 bits: 65536) Higher sampling, better quantization means better sound, but more space Higher sampling, better quantization means better sound, but more space Chosen sampling rate, quantization, and compression techniques rely on human perceptual capabilities (psychoacoustics) Chosen sampling rate, quantization, and compression techniques rely on human perceptual capabilities (psychoacoustics)