1. CS 423 (CS 423/CS 523) Computer Vision Lecture 1 INTRODUCTION TO COMPUTER VISION

2. About the Course 2

3. http://vvgl.ozyegin.edu.tr Objective Introduction to the theory, tools, and algorithms of computer vision Instructor Assist. Prof. M. Furkan Kıraç E-mail: furkan.kirac@ozyegin.edu.tr Room: 219 Hours Mondays, 9:40-12:30, Room: 246 Grading Projects: 4x15% Midterm Exam: 40% Syllabus 3

4. Projects: Late submissions are not accepted. Copying answers from others’ work is not permitted. Midterm Exam: At least 3 of the 4 Projects must be turned in by the due date in order to qualify for the Final Exam. No Composite Exam (Bütünleme Sınavı), as there is no final exam. Grading 4

5. Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010. Computer Vision: A Modern Approach, David A. Forsyth and Jean Ponce, Prentice-Hall, 2002. Introductory Techniques for 3D Computer Vision, Emanuele Trucco and Alessandro Verri, Prentice-Hall 1998. Recommended Books 5

6. OpenCV Computer Vision Application Programming Cookbook Second Editon, Robert Laganière, Packt Publishing, 2014. Learning OpenCV, Gary Bradski and Adrian Kaehler, O'Reilly, 2008. Mastering OpenCV with Practical Computer Vision Projects, Daniel Lélis Baggio, et al., Packt Publishing, 2012. OpenCV Resources 6

7. Applications of Computer Vision 7

8. Image Stitching

9. Image Matching

10. Object Recognition

11. 3D Reconstruction

12. Interior Modeling 12

13. 3D Augmented Reality 13

14. 3D Camera Tracking 14

15. 15 Stereo Conversion for 3DTV

16. Depth Estimation and View Interpolation for 3DTV 16

17. Human Tracking 17

18. License Plate Recognition 18

19. Human Pose Estimation 19

20. Course Outline 20

21. Linear Filters, Frequency Domain Filtering, Edge and Boundary Detection Feature Detection Fitting, Alignment Histograms Covariance, Principle Component Analysis (PCA) Face Detection and PCA Optical Flow and Motion Tracking and Mean-Shift Randomized Decision Trees, Pose Estimation Bag of Features Context, Two-View Geometry Summary Topics to be covered... 21

22. Relation to Other Fields 22

23. Computer Vision 23 Figure from "Computer Vision: Algorithms and Applications,” Richard Szeliski, Springer, 2010.

24. Lights and materials Shading Texture mapping Environment effects Animation 3D scene modeling 3D character modeling (OpenGL) Computer Graphics 24

25. Computer Graphics 25

26. Resampling Enhancement Noise filtering Restoration Reconstruction Segmentation Image compression (MATLAB and OpenCV) Image Processing Topics 26

27. Image Processing 27

28. Motion estimation Frame-rate conversion Multi-frame noise filtering Multi-frame restoration Super-resolution Video compression (MATLAB & OpenCV) Video Processing Topics 28

29. Video acquisition-display chain 29 CaptureRepresentationCoding TransmissionDecodingRendering

30. Human vs. Computer 30

31. Optical illusions

32. Actual vs. Perceived Intensity (Mach band effect) 32

33. Brightness Adaptation of the Eye 33

34. Optical illusions

37. Why is Computer Vision Difficult?

38. Human perception

40. Human Visual System 40

41. Human Eye

44. Photoreceptors: Rods & Cones

46. Rods vs. Cones Rods Perceive brightness only Night vision Cones Perceive color Day vision Red, green, and blue cones

47. Cone Distribution 64% 32% 2% Blue is less-focused

48. Visual Threshold drop during Dark Adaptation

49. Spatial Resolution of the Human Eye Photopic (bright-light) vision: Approximately 7 million cones Concentrated around fovea Scotopic (dim-light) vision Approximately 75-150 million rods Distributed over retina (HDTV: 1920x1080 = 2 million pixels) 49

50. Frequency Responses of Cones Same amount of energy produces different sensations of brightness at different wavelengths Green wavelength contributes most to the perceived brightness. 50

51. Trichromatic Color Mixing Any color can be obtained by mixing three primary colors Red, Green, Blue (RGB) with the right proportion

53. Image Formation 53

54. Human Eye vs. Camera Camera componentsEye components LensLens, cornea ShutterIris, pupil FilmRetina Cable to transfer imagesOptic nerve to send the incident light information to the brain

55. Human Vision

56. Image formation

57. Pin-Hole Camera Model

58. Point Spread Effect

59. Out-of-Focus Blur

60. Shrinking the Aperture

61. Converging Lens

62. Correction with a Converging Lens

64. Perfectly In-Focus for a Certain Distance Only “circle of confusion”

65. Depth-of-Field

67. “Sharp Image” within Depth-of- Field due to Finite Sensor Size

68. Focal Length ( F ) and Depth ( Z )

69. Aperture Size Affects Depth-Of-Field f / 5.6 f / 32

70. Aperture

71. Camera f-number

72. Exposure Time

73. Motion Blur Effect due to Finite Exposure Time

74. Decrease in aperture implies… Increase in depth-of-field Decrease in motion blur Decrease in exposure

75. 2D Image Representation 75

76. 76 Image Capture (Courtesy Gonzalez & Woods)

77. Digital Image Capture

78. Light sensitive diodes convert photons to electrons

79. Color Image Capture: Single vs. Three CCD Arrays RGB splitter (three separate imaging sensors, higher resolution) Bayer filter (cheaper but introduces spatial resolution loss)

80. Digital Camera Issues Noise caused by low light Color color fringing (chromatic aberration) artifacts from Bayer patterns Blooming charge overflowing into neighboring pixels In-camera processing over-sharpening can produce halos Compression creates blocking artefacts

81. Digitization: Sampling and Quantization

82. Sampling Rate Problem

83. Over Quantization 83

84. 84 Images as Matrices of Integers 126127126 125126127 123126125 128127124 123120144 121128155 126123127 120122124 119121123 122142162 130157161 145162164 158 163 160 164 166 165 m n (0,0) 0 ≤ s(m,n) ≤ 255 } quantization 0 ≤ m ≤ M-1 0 ≤ n ≤ N-1 MxN 8-bit gray-scale (intensity, luminance) image sampling 0 → black, 255 → white

85. Images as Functions We can think of an image as a function, f, from R 2 to R: f( x, y ) gives the intensity at position ( x, y ) Realistically, we expect the image only to be defined over a rectangle, with a finite range: f: [a,b] x [c,d]  [0,1] A color image is just three functions pasted together. We can write this as a “vector-valued” function:

86. RGB Color Bands (Channels)

87. YUV Bands Also called Y Cb Cr Y : Luma Cb : Chrominance_blue Cr : Chrominance_red Y U (Cb) V (Cr ) Color

88. YUV-RGB Conversion

89. Summary 89

90. Human visual system Pin-hole camera model Image representation Summary 90