1. Lecture outline Basics Spectral image Spectral imaging systems Applications Summary Lecture material by: Markku Hauta-Kasari, Kanae Miyazawa, Jussi Parkkinen, Timo Jääskeläinen, Jouni Hiltunen, Joni Orava, Hannu Laamanen, Jarkko Mutanen

3. Spectral measurement

6. Human cone sensitivities

8. Chicken cone sensitivities

9. Spectral approach to color In spectral approach, color is represented by color signal. This causes the color sensation The signal is part of electromagnetic spectrum - in human color vision the range is 380-780 nm In spectral approach, we are not limited into this human visual range

10. Motivation for spectral color Not to loose important color information (To avoid the problem of metamerism) To define optimal color sensors To develop better color vision models To develop novel instruments To develop spectral color classifiers and optical implementations for them

12. Outline Basics Spectral image Spectral imaging systems Applications Summary

13. Component images of spectral image

14. Spectral Image

16. Spectra from leaves in previous image

17. MEMORY REQUIREMENTS OF IMAGES Image size256x256 512x512 gray-level image 65 kb 262 kb color (RGB-) image 196 kb 786 kb spectral, 20 nm resol. 1 Mb 4 Mb spectral, 5 nm resol. 3 Mb 15 Mb

18. Definitions Spectral image An image, where each pixel is represented by a spectrum Hyperspectral A term used for spectral images with large number of spectral components RGB-image spectrum in visible region, three components Multispectral

19. Image Types TYPE SPECTRAL COMPONENTS --------------------------------------- Gray-scale Trichromatic Spectral –Hyperspectral Real-time spectral Single Three >3 Numerous

20. Outline Basics Spectral image Spectral imaging systems Applications Summary

21. Spectral Imaging Devices Spectral cameras –filter wheels –light filtering –scanning systems –multi-band detectors

22. Optical principles Narrow band filters –interference filters, LCTF, gratings –AOTF Broad band filters –absorbance filters –e.g. gratings to implement optimal filters

23. Joensuun yliopisto PL 111 80101 Joensuu puh. (013) 251 111 fax (013) 251 2050 www.joensuu.fi Jan 02/tj Formation of the Color Signal

25. Filter wheel based system

26. Specim Spectral Camera http://www.specim.fi

27. A spectrum sampled at 39 wavelengths

28. Spectral Imaging One approach: to measure the spectral data accurately  A large amount of data Other approach: to measure component images using a few optimally designed color filters  Data is convenient for storing and transmission  Spectral image can be reconstructed computationally

29. Color Filter Design One approach: to choose an optimized set of commercially available color filters (for example, Kodak Wratten gelatin filters) Other approach: to design optimal color filters computationally (our approach)  adaptive to various application  rewritable filter based imaging system needed  Spectral image can be reconstructed computationally, if needed

30. ACTIVE TYPE Optimal Light Source Sample CCD camera CCD camera Outdoor Indoor Light Source Optimal Filter Sample + Optimal Light Source Sample + Optimal Filter PASSIVE TYPE Next, computational color filter design and the following spectral imaging systems will be studied

31. Outline Basics Spectral image Spectral imaging systems Applications Summary

32. Some applications E-commerce Tele-medicine Image archives Accurate color measurement and representation

33. Digital Imaging

34. Display characterictics

36. Broadband network society Image Processing System Display Image Acquisition Network Database

37. E-commerce Telemedicine E-commerce, telemedicine in broadband network society Clothes Paints Textile Bags …. Facial color Skin disease Expressions …..

38. Joensuun yliopisto PL 111 80101 Joensuu puh. (013) 251 111 fax (013) 251 2050 www.joensuu.fi Jan 02/tj E-commerce Differences between images on a display and real images Online shopping Return of products Figure: Tsumura-san, Chiba Univ.., Japan

39. Joensuun yliopisto PL 111 80101 Joensuu puh. (013) 251 111 fax (013) 251 2050 www.joensuu.fi Jan 02/tj Environmental dependence of color Network Patient Medical doctor

40. Image reproduction multi-primary displays - six components projective display (Natural Vision Research Center, Tokyo) multi-primary printing - inkjet

41. Multiprimary display The objects are measured as spectral images The display contains 6 primary colors  avoids the problem of metamerism  larger color gamut  natural colors  colors can be seen the same in the measurement place and in the viewing place

45. RGB-projector

46. RGB-filters High and low pass filters Multiprimary display 6 filters for

47. Modified RGB-projector

48. Multiprimary display

49. Color gamuts for CRT-display and multiprimary display

50. Spectral video Sequence of spectral images Shown as movie on the screen Very high memory requirements Efficient compression has been used Shown as RGB or multiprimary (TAO, Japan)

51. Compression method it has been found that separate spatial and spectral compression give best results in spectral dimension, PCA-based compression give best results combination of PCA in spectral and JPEG in spatial dimension give best PSNR- values for reasonable compression ratios

52. Outline Basics Spectral image Spectral imaging systems Imaging context Applications Summary

53. Spectral imaging increasing Novel instruments needed Novel detectors needed Novel image compression methods needed Color research at Joensuu: WWW: http://cs.joensuu.fi/~spectral Email: mhk@cs.joensuu.fi

54. Reconstructed Images Reconstructed Images from Measured Spectral Images

55. Observation of Spectral Reflectance

56. Church

57. Color Change by Cleaning OriginalOnceTwice Three Times Reconstructed images from Measured Spectral Images

58. Observation of Spectral Reflectance Original Once Twice Three Times

59. Cleaning Process 1 st Cleaning2 nd Cleaning3 rd Cleaning Each subtract image is normalized by Maximum value