1. Sensory Information Processing Introduction to imaging sensors

2. Camera CCD driver / Signal processing Digital Capture CCD lens What is principal parameter of the lens? What is aberration? What is the image degradation by lenses? How to select the lenses? How the image is recorded electrically? What is the artifact of CCD? How to select the CCD?

3. Camera How to drive the CCD? How the color value is recovered? How the Synchronize and Sampling is performed? CCD driver / Signal processing Digital Capture CCD lens

4. Measuring the light History of light measurement –Exposure meter for cameras (Human can not percept absolute amount of the light) Leica MC meter (1954) Contaflex(1935) : world’s first camera with exposure meter

5. Before that.. Rely on inspiration and experience.. Senoguide

6. Device to measure the amount of light Transducing energy of light to electricity –Photo diode –Solar cell (similar to photo diode) Change of resistance according to the light –CdS (sulfureted cadmium) cell –Photo tube Light  displacement, speed, etc.. –So weak, very difficult

7. CdS cell Change of resistance of sulfureted cadmium (compound semiconductor) –Free electrons are emerged by the light, and conducts electric current well –Wide range of sensitivity –Slow reaction –cheap Inadequate for image sensor

8. CdS cell(2) Winding CdS area –Wider width, shorter path Burn out ・ very good linearity electrode Relative sensitivity Wavelength(nm) Electric current (mA) Maximum power consumption 100mW Voltage (V)

9. phototube(1) Photoelectric effect –electron is kicked off from metal by photon metal photon e-e- Characteristics of photoelectric effect –No electron for the light with longer wavelength than threshold which depends on materials –Number of electron is proportional to the intensity of light

10. phototube(2) Capturing the electron out from the metal –Material: Ag, Cs, Na, Te, Ga, As –Very high speed of response : up to 10ps Electron is attracted by electric potential by high voltage

11. Phototube(3) Very high sensitivity (photomultiplier) Dynodes (SbCs, AgHg etc.) emit lots of electron by hitting the accelerated electron ． http://en.wikipedia.org/wiki/Photomultiplier

12. photomultiplier(4) Super-kamiokaNDE (observing facility of neutrino) http://en.wikipedia.org/wiki/Kamiokande

13. Broken photomultiplier of kamiokaNDE

14. photodiode(1) Major light sensing device –Photoelectron emerged between p and n semiconductor –No vaccum!

15. history of exposure meter Solar cell type : require no battery – １９５０ CdS type : high sensitivity (but slow) – １９６０ Photodiodes – １９７０ Leica MR meter Leica MC meter Current products

16. 2-D image sensor How to capture the 2-D distibution of light? Array of sensors? Connecting all sensors each?

17. 2-D image sensors To reduce the wiring cost, scanning mechanism is necessary Electric signal

18. Camera tube Magnetic field is used to deflection (scanning)

19. Camera and TV tubes Magnetic field for scanning

20. Image orthicon

21. CCD (charge coupled device) Electrostatic effect is used to transfer the electron –Not limited to the imaging device –CCD is the term of the way to transfer the electron –http://en.wikipedia.org/wiki /Charge-coupled_devicehttp://en.wikipedia.org/wiki /Charge-coupled_device

22. Aside : delay line (1) How to delay temporal signal? –“dead time component”

23. Aside : delay line (1)

24. Delay line is a memory –Recording information for a short time –No sampling, no quantization

25. Delay line (2) SAW (surface acoustic wave) filter –http://en.wikipedia.org/wiki/Surface_acoustic_wavehttp://en.wikipedia.org/wiki/Surface_acoustic_wave

26. Delay line (3) BBD BBD = Bucket Brigade delay Predecessor of CCD MN3004

27. Delay line (4) current standard Long time delay for sound, video is based on digital technology AD converterCPUDA converter RAM

28. CCD types Full frame Interline transfer Frame transfer Frame interline transfer http://laser.physics.sunysb.edu/~jose/CCD Info.htmlhttp://laser.physics.sunysb.edu/~jose/CCD Info.html CCD university –http://www.ccd.com/ccdu.htmlhttp://www.ccd.com/ccdu.html

29. Full frame While transfer, CCD should be covered (mechanical shutter is necessary) High aperture ratio Easy to manufacture Commonly used in digital SLR

30. Interline transfer Most popular Mechanical shutter is not necessary Very short exposure is possible Used in most video cameras

31. Frame transfer Image is transferred to the storage array quickly

32. Image degradation in CCD smear –Highlight spreading to the transfer direction (vertical) blooming –Highlight spreading to the peripherals Thermal noise –Electron is emerged by the thermal agitation of free electron

33. smear

34. Source of smear Imperfect mask of transfer CCD areas, spilling out of electron to the transfer area mask

35. Anti-blooming Discarding too much electrons by highlight

36. X-Y address image sensor CMOS image sensor photodiodes are connected by wire and transistor CMOS transistor is used, therefore, commonly called as CMOS sensor JFET type

37. CMOS image sensor Low power consumption –CCD needs high voltage to attract the electron Other circuit can be embedded on a chip (usual LSI process is used) –AD converter, image processor can be embedded Noise was more than CCD –Now it is well improved Flexibility to access to each pixel

38. Rolling shutter For most CMOS sensors, exposure timing is varied pixel by pixel –Called “rolling shutter” –If all pixels are exposed simultaneously, it is called “global shutter”

39. Shutter types and image skew Lens shutter （ interchanging the lens is not easy) Focal plane shutter （ suit for interchanging lens) SLR (single lens reflex) etc.

40. Skew of moving object Shape of the object is skewed by the varied timing of exposure

41. Ancient example of skew Lartigue “A.C.F grand prix” World’s most famous example of skew

42. Color image Sequencial shots of R, G, B –filter wheel Split R, G, B to three image sensors optically – ３ tubes, 3-CCD Each pixel is sensitive either R, G or B –Primal color, complementary color –Foveon

43. Filter wheel

44. ３ tubes lens prism Camera tube

45. ３ -CCDcamera Each surface in the prism is “interference filter” which consist of transparent materials with different refractive index, and interference of light reflect or transmit the energy of the light without loss. No loss of light energy, high sensitivity Color reproduction is good

46. Single CCD Each pixel of CCD is colored –Primal color ○natural color reproduction –Complementary color ○high light efficiency RG GB Primal color （ Bayer pattern) CyYeCyYeCyYeCyYe MgG G G G Complementary color RG GB RG GB RG GB RG GB RG GB RG GB RG GB RG GB RG GB RG GB RG GB Mg G CyYe MgG CyYe MgG CyYe MgG CyYe CyYeCyYeCyYeCyYe Mg G G G G

47. Color reproduction (2) Simple method (interpolation) See Fredo Durand and Bill Freeman’s slide –http://groups.csail.mit.edu/graphics/classes/CompPho to07/index2.htmlhttp://groups.csail.mit.edu/graphics/classes/CompPho to07/index2.html –03_colorInterpCP.ppt G G G G G G G G G G G G G G G G G G G G G G G G RRRR RRRR RRRR Low resolution