1. BPC: Art and Computation – Fall 2006 Digital Media II: Light, Vision & Digital Images Glenn Bresnahan glenn@bu.edu

2. BPC: Art and Computation – Fall 20062 Outline What is light? Properties of light How do we see? Digital representation of images Computer display Digital image formats

3. BPC: Art and Computation – Fall 20063 How Do We See?

4. BPC: Art and Computation – Fall 20064 How Do We Hear? Sound waves move through the air Waves interact (e.g. reflect) w/ environment Sounds wave reach our ear

5. BPC: Art and Computation – Fall 20065 How Do We See? Light is emitted from a source Waves interact (e.g. reflect) w/ environment Light reaches our eyes

6. BPC: Art and Computation – Fall 20066 What is Light Light is a wave Packets of light energy are called photons

7. BPC: Art and Computation – Fall 20067 Waves Revisited

8. BPC: Art and Computation – Fall 20068 Waves – Properties Amplitude Wavelength (distance)

9. BPC: Art and Computation – Fall 20069 Waves in Motion – Properties Period (time for one cycle) Time 1 2 Frequency cycles per time interval

10. BPC: Art and Computation – Fall 200610 Cycles and Circles Sine waves and circles are closely related Y axis X axis angle (x,y)

11. BPC: Art and Computation – Fall 200611 Cycles and Circles Y axis X axis angle (x,y)

12. BPC: Art and Computation – Fall 200612 Properties of Sound Pitch is the perception of frequency Human perception: 20 Hz – 20 KHz Sound travels at approx. 1100 feet/second in air –Approx. 750 miles/hour or 1 mile every 4.8 sec. Loudness perception of amplitude

13. BPC: Art and Computation – Fall 200613 Properties of Light Color is the perception of frequency Human perception: 430 – 750 THz (red – violet) –1 THz = 1,000,000,000,000 Hz Light travels at approx. 186,000 miles/second in air –Approx 1 foot every nanosecond Brightness is perception of energy level (number of photons)

14. BPC: Art and Computation – Fall 200614 How Fast is Light? 186,00 miles/sec or 300,000 meters/sec 8 minutes to reach earth from sun

15. BPC: Art and Computation – Fall 200615 Wavelength Wavelength = Speed / Freq –E.g. 1 ft/sec at 1 Hz = 1 ft wavelength –Higher frequencies == shorter wavelengths Red = 300KM/Sec / 430 THz = 698 nm (nano (billionth) meters) Violet = 300KM/Sec / 750 THz = 400 nm

16. BPC: Art and Computation – Fall 200616 Visible Spectrum Where is the white light? What happens at higher/lower frequencies?

17. BPC: Art and Computation – Fall 200617 Electromagnetic Spectrum Visible light is electromagnetic force in a particular frequency range

18. BPC: Art and Computation – Fall 200618 Light Interaction with Materials When light hits a surface, several things can happen. The light can be: –Absorbed by the surface Converted to another form of energy –Reflected (bounced) off the surface –Transmitted (refracted) through the surface

19. BPC: Art and Computation – Fall 200619 Absorption and Reflection Different materials will absorb different frequencies The absorption vs. reflection determines the color of the material –Black materials absorbs all wavelengths –White material reflects all wavelengths –Blue material reflects blue and absorbs all other wavelengths Combining pigments causes more wavelengths to be absorbed, fewer wavelengths to be reflected –Subtractive color

20. BPC: Art and Computation – Fall 200620 Reflection and Refraction

21. BPC: Art and Computation – Fall 200621 Reflection Light reflects at an opposite and equal angle –Specular (mirror) reflection Some light will be scattered in all directions

22. BPC: Art and Computation – Fall 200622 Refraction Speed of a wave varies by material Index of refraction is relative speed in the medium –Vacuum1.0000 –Air1.0003 –Ice1.31 –Water1.33 –Quartz1.46 –Flint glass1.57-1.75 –Diamond2.417

23. BPC: Art and Computation – Fall 200623 Refraction When a wave chances speed it changes direction, i.e. bends The angle depends of the change in refractive index

24. BPC: Art and Computation – Fall 200624 Refraction Objects appear to bend in water

25. BPC: Art and Computation – Fall 200625 Refraction Lens change size of objects

26. BPC: Art and Computation – Fall 200626 Combination of Light White light? –Combination of multiple colors (freq) of light What happens when we combine different frequencies of light, say red and green? What happens when we combine different frequencies of sound, say an C and an E note?

27. BPC: Art and Computation – Fall 200627 Color Experiment If we combine red, green and blue light we get new colors in the region of overlap Colors seem to “add”

28. BPC: Art and Computation – Fall 200628 How We See Light is emitted from a source Light interacts with surfaces in the environment Light is reflected into our eyes

29. BPC: Art and Computation – Fall 200629 Human Vision Light passes into the cornea, though a liquid filled chamber and out through the lens. These focus the light The pupil acts as diaphragm, controlling the amount of light The light is projected onto the retina at the back of the eye

30. BPC: Art and Computation – Fall 200630 Human Vision The retina is covered with photosensitive receptor cells Photoreceptor cells are attached to the optical nerve which feeds signals to the brain Light (photons) enter the cell cause a chemical reaction which causes the cell to fire

31. BPC: Art and Computation – Fall 200631 Eat Your Carrots! Photoreceptor cells contain opsin (a protein) + retinal = rhodopsin Photo excitation causes the rhodopsin to twist and release the retinal The released retinal causes a reaction which cause the attached nerve to fire Retinal is destroyed in the process Retinal is synthesized from vitamin A Vitamin A is derived from beta- carotene

32. BPC: Art and Computation – Fall 2006 Digital Media II: Light, Vision & Digital Images Part 2 Glenn Bresnahan glenn@bu.edu

33. BPC: Art and Computation – Fall 200633 Question? When we combine light of two different frequencies we seem to get light of a different color. Why does this happen? Sound waves don’t combine this way.

34. BPC: Art and Computation – Fall 200634 Combining Waves Sound waves do not combine to make new frequencies (pitch) C + E not equal D C = 523.25 Hz / 65.9 cm (2.162 ft) D = 587.33 Hz / 58.7 cm (1.925 ft) E = 659.29 Hz / 52.3 cm (1.716 ft)

35. BPC: Art and Computation – Fall 200635 Length of Light Waves Human hair ~ 1/500” –0.005 cm –50,000 nm Cyan light = 500 nm 100 wavelengths across a human hair

36. BPC: Art and Computation – Fall 200636 Human Vision Light passes into the cornea, though a liquid filled chamber and out through the lens. These focus the image The pupil acts as diaphragm, controlling the amount of light The light is projected onto the retina at the back of the eye where a chemical reaction causes neurons to fire

37. BPC: Art and Computation – Fall 200637 Photoreceptors The retina contains two types of receptor cells: rods and cones Approx. 90 million rods; 4.5 million cones

38. BPC: Art and Computation – Fall 200638 Photoreceptors - Rods Rods react to very low light levels –As few as several photons Rods react to a broad spectrum of frequencies (max at 498 nm) Rods react slowly (~100 milliseconds)

39. BPC: Art and Computation – Fall 200639 Photoreceptors - Cones Cones require much more light to fire Cones react much more quickly (10- 15 ms) Cones are much denser in the center (fovea) of the eye

40. BPC: Art and Computation – Fall 200640 Photoreceptors – Distribution

41. BPC: Art and Computation – Fall 200641 Photoreceptors - Cones Three types of cones: S, M, L which react to different wavelengths of light –L Cones: peak at 564 nm –M Cones: peak at 533 nm –S Cones: peak at 437 nm

42. BPC: Art and Computation – Fall 200642 Photoreceptors – Response Spectrum S = blue, M = green, L = red

43. BPC: Art and Computation – Fall 200643 Photoreceptors – Seeing Colors Any response can be synthesized by combining red, green and blue light

44. BPC: Art and Computation – Fall 200644 Color Mixing Adding red, green and blue light in various proportions can generate the perception of all colors

45. BPC: Art and Computation – Fall 200645 Cell Firings Light reaching photoreceptors causes some number of cells to fire (after an interval) Cells can not continually fire Receptors can become saturated Cell firings are discrete

46. BPC: Art and Computation – Fall 200646 Saturation – After Images

47. BPC: Art and Computation – Fall 200647 Saturation – After Images

48. BPC: Art and Computation – Fall 200648 Flicker Fusion If light is flashed fast enough, it becomes indistinguishable from a steady light The rate is called the flicker fusion or critical flicker frequency Dependent on intensity, but about 45 Hz

49. BPC: Art and Computation – Fall 200649 Flicker Fusion & Animation Flicker fusion makes animation possible Each frame is displayed a fraction of a second

50. BPC: Art and Computation – Fall 200650 Flicker Fusion in Film and Video Film uses 24 frames per second Video uses 30 frames per second Flicker fusion is >45 FPS How does this work??

51. BPC: Art and Computation – Fall 200651 Flicker Fusion in Film Film projector has a shutter Each frame is displayed 3 times

52. BPC: Art and Computation – Fall 200652 Flicker Fusion in Video Frame is broken up into strips (scan lines) Frame is divided into two fields: odd lines and even lines Fields are displayed at 60 Hz

53. BPC: Art and Computation – Fall 200653 Seeing The rods and cones cause nerves to fire and electrical signals to be send to the brain. There are ~100 million receptor cells generating impulse streams Impulses are combined by other nerve cells 1.2 million nerve fibers in optic nerve bundle

54. BPC: Art and Computation – Fall 200654 Vision Is Complicated

55. BPC: Art and Computation – Fall 200655 Digital Images

56. BPC: Art and Computation – Fall 200656 Digital Images Red= 100% Green = 80% Blue = 60%

57. BPC: Art and Computation – Fall 200657 Digital Images Image is constructed from a grid (array) of individual color dots Individual dots are called pixels (picture elements) Resolution is the number of elements in each direction (e.g. 1280 in x, 1024 in y = 1.3 Mpixel) Each pixel is composed of three color components representing levels of R,G,B light Each level (R,G,B) is represented by a number –One byte (0-255) per component, e.g. 255,204,153 The array of pixel values is stored in a graphics frame buffer (memory) The pixel values are read out (at approx. 60 fps) and used to display the image on a monitor

58. BPC: Art and Computation – Fall 200658 Graphics Display RGB RGB RGB … … … … Frame Buffer Computer Monitor

59. BPC: Art and Computation – Fall 200659 Cathode Ray Tube Display Same technology as a TV screen Electron beam is aimed at the screen When the beam hits a phosphor on the surface it glows Three different colored (RGB) beams phosphors

60. BPC: Art and Computation – Fall 200660 Other Displays Liquid Crystal (LCD) Plasma panel Digital Light Projection

61. BPC: Art and Computation – Fall 200661 Digital Storage of Images Need to store RGB value for each pixel 1024x1280 pixels = 3.9 million numbers Different images files use different ways of storing the numbers –Most file formats store additional information –Formats vary in how much information per pixel is stored Some formats compress the information –Compression can lead to loss of detail –Uncompressing the compressed image is NOT the same as the original image. –Repeatedly storing (compressing) and retrieving (uncompressing) can cause an image to degrade