Presentation is loading. Please wait.

Presentation is loading. Please wait.

3D Computer Vision: CSc 83020.

Similar presentations

Presentation on theme: "3D Computer Vision: CSc 83020."— Presentation transcript:

1 3D Computer Vision: CSc 83020

2 3D Computer Vision: CSc 83020 Instructor: Ioannis Stamos
istamos (at) Office Hours: Tuesdays 4-6 (at Hunter) or by appoitnment Office: 1090G Hunter North (69th street bw. Park and Lex.) Computer Vision Lab: 1090E Hunter North Course web page:

3 Goals To familiarize you with basic the techniques and jargon in the field To enable you to solve computer vision problems To let you experience (and appreciate!) the difficulties of real-world computer vision To get you excited!

4 Class Policy You have to Late policy Teaming
Turn in all assignments (60% of grade) Complete a final project (30% of grade) Actively participate in class (10% of grade) Late policy Six late days (but not for final project) Teaming For final project you can work in groups of 2

5 About me 11th year at Hunter and the Graduate Center
Graduated from Columbia in ’01 CS Ph.D. Research Areas: Computer Vision 3D Modeling Computer Graphics

6 Books Computer Vision: Algorithms and Applications, Richard Szeliski, 2010 (available online for free) Robot Vision B. K. P. Horn, The MIT Press (great classic book) Introductory Techniques for 3-D Computer Vision Emanuele Trucco and Alessandro Verri, Prentice Hall, 1998 (algorithmic perspective) Computer Vision A Modern Approach David A. Forsyth, Jean Ponce, Prentice Hall 2003 An Invitation to 3-D Vision Yi Ma, Stefano Soatto, Jana Kosecka, S. Shankar Sastry Springer 2004. Three-Dimensional Computer Vision: A Geometric Viewpoint Olivier Faugeras The MIT Press, 1996.

7 Journals/Web International Journal of Computer Vision.
Computer Vision and Image Understanding. IEEE Trans. on Pattern Analysis and Machine Intelligence. SIGGRAPH (mostly Graphics) (CMU’s Robotic Institute) (The Vision Home Page) (CV Online) (Annotated CV Bibliography)

8 Class History Based on class taught at Columbia University
by Prof. Shree Nayar. New material reflects modern approach. Taught similar class at Hunter Taught “3D Photography” class at the Graduate Center of CUNY. My active research area Funded by the National Science Foundation

9 Class Schedule Check class website Final project proposals
Due Nov. 7 Design your own or check list of possible projects on class website Final project presentations and report May 16 (last class)

10 What is Computer Vision?
Sensors Images or Video Illumination Vision System Physical 3D World Scene Description Measuring Visual Information

11 Computer Graphics Output Image Model Synthetic Camera
(slides courtesy of Michael Cohen)

12 Computer Vision Output Model Real Scene Real Cameras
(slides courtesy of Michael Cohen)

13 Combined Output Image Model Real Scene Synthetic Camera Real Cameras
(slides courtesy of Michael Cohen)

14 Cont. Vision is automating visual processes (Ball & Brown).
Vision is an information processing task (Marr). Vision is inverting image formation (Horn). Vision is inverse graphics. Vision looks easy, but is difficult. Vision is difficult, but it is fun (Kanade). Vision is useful.

15 Some Applications Industrial Material Handling Inspection Assembly

16 Some Applications Autonomous Navigation

17 Some Applications Vision for Graphics Film Industry Urban Planning
E-commerce Virtual Reality

18 Some Applications Realistic 3D experience Google Earth
Microsoft Photosynth

19 More Applications! Optical Character Recognition (OCR)
Visual Databases (images or movies) Searching for image content Face Recognition (security) Iris Recognition (security) Traffic Monitoring Systems Many more…

20 Vision deals with images

21 Images Look Nice…

22 Images Look Nice… Ioannis Stamos – CSc Spring 2007

23 ...Essentially a 2D array of numbers

24 Low-Level or “Early” Vision
Considers local properties of an image “There’s an edge!” From: Szymon Rusinkiewicz, Princeton.

25 “There’s an object and a background!”
Mid-Level Vision Grouping and segmentation “There’s an object and a background!”

26 High-Level Vision Recognition “It’s a chair!”

27 Humans Vision is easy for us. But how do we do it?

28 Human Vision: Illusions
Fraser’s spiral (Fraser 1908)

29 Illusions Zölner Illusion (1860) Hering Illusion (1861)
Wundt Illusion (1896)

30 Visual Ambiguities Young-Girl/Old-Woman

31 Visual Ambiguities From NALWA.

32 Visual Ambiguities

33 Visual Ambiguities Two lava cones with craters, perceived to be the image of two craters with mounds. Implicit assumption that lighting is from above.

34 Visual Ambiguities

35 Seeing and Thinking Kanizsa (1979)

36 Syllabus Overview

37 Image Formation and Optics
Light Source p Surface normal CCD Array Lens P Object Surface Projection of 3-D World on a 2-D plane

38 Lenses Ray of light Optical Axis

39 Image Sensors/Camera Models
Typical 512x512 CCD array Imaging Area 262,144 pixels One Pixel 20μm 20μm 512 (10.25mm) Convert Optical Images To Electrical Signals. 512 (10.25mm)

40 Filtering = h g f

41 Image Features Detecting intensity changes in the image
Ioannis Stamos – CSc Spring 2007

42 Grouping image features
Finding continuous lines from edge segments Ioannis Stamos – CSc Spring 2007

43 Camera Calibration Camera Coordinate Frame Zc Pixel Coordinates Yc
Intrinsic Parameters Xc Extrinsic Parameters Image Coordinate Frame Zw Yw World Coordinate Frame Xw

44 Shape from X Shape from X Stereo Motion Shading Texture foreshortening

45 Binocular Stereo depth map

46 Active Sensing Sheet of light CCD array Lens Sources of error:
1) grazing angle, 2) object boundaries. Sheet of light CCD array Lens

47 Shape from Shading Three-dimensional shape from a single image.
Ioannis Stamos – CSc Spring 2007

48 Motion (optical flow) Determining the movement of scene objects
Ioannis Stamos – CSc Spring 2007

49 Reflectance and Color Why do these spheres look different?

50 Object Recognition Learning visual appearance.
Real-time object recognition.

51 Template-Based Methods
Cootes et al.

52 Some Vision Systems…

53 Example 2: Structure From Motion
Slide courtesy of Sebastian Thrun Stanford

54 Example 2: Structure From Motion
Slide courtesy of Sebastian Thrun Stanford

55 Example 2: Structure From Motion
Slide courtesy of Sebastian Thrun Stanford

56 Example 2: Structure From Motion
Slide courtesy of Sebastian Thrun Stanford

57 Example 2: Structure From Motion
Slide courtesy of Sebastian Thrun Stanford

58 Example 4: 3D Modeling Slide courtesy of Sebastian Thrun
Stanford Drago Anguelov

59 Example 5: Segmentation
Slide courtesy of Sebastian Thrun Stanford

60 Example 6: Classification
Slide courtesy of Sebastian Thrun Stanford

61 Example 6: Classification
Slide courtesy of Sebastian Thrun Stanford

62 Real-world Applications
Osuna et al:

63 Range Scanning Outdoor Structures
Ioannis Stamos – CSc Spring 2007

64 Data Acquisition Spot laser scanner. Time of flight. Max Range: 100m.
Scanning time: minutes for x1000 points. Accuracy: 6mm.

65 Video

66 Latest Video

67 Inserting models in Google Earth

68 Image sequence (CMU, Virtualized Reality Project)
Dynamic Scenes Image sequence (CMU, Virtualized Reality Project)

69 Dynamic Scenes Dynamic 3D model.

70 Dynamic texture-mapped model.
Dynamic Scenes Dynamic texture-mapped model.

71 Scanning the David Marc Levoy, Stanford
2:30 through closeup of eye 0:40 So here we are scanning our “piece de resistance” the gantry is extended to nearly its full height If you’ve looked at our web site you know the little story about the gantry being too short initially because we relied on art history books for the height of the David and they're all wrong – by 3 feet !! so at the last minute we had to add a piece to the gantry…here and load another few hundred pounds of ballast to the base…here – to keep the gantry from tipping over on the statue But as you can see from the photo on the right, we did finally reach the top of the statue height of gantry: meters weight of gantry: kilograms 4:00 through architectural reps 0:45

72 Statistics about the scan
0:20 So here’s a rendering of our computer model of the David we rolled the gantry, which now weighed about a ton, 480 times 2 billion polygons you can read the rest It took 30 days to scan the statue but we were only allowed to scan at night, when the museum was closed so it was basically 30 all-nighters in a row 480 individually aimed scans 2 billion polygons 7,000 color images 32 gigabytes 30 nights of scanning 22 people Ioannis Stamos – CSc Spring 2007 0:30

73 Head of Michelangelo’s David
0:20 We haven’t assembled the David at full-resolution yet Here’s a 1 mm model of the head watertight with color The photograph at the left was taken with uncalibrated lighting so the two images don’t match exactly but hopefully you agree that we’ve basically captured the statue’s appearance photograph 1.0 mm computer model

74 David’s left eye 0.25 mm model photograph
0:30 Here’s a closeup of David’s left eye at the maximum resolution of our model – ¼ mm we can see some interesting things… These bumps are holes from Michelangelo’s drill so that’s real geometry are artifacts from space carving, the hole-filling part of our range image merging algorithm smoothing out those artifacts, while preserving the observed surfaces, is future work Finally, these bumps are noise from laser subsurface scattering Let’s zoom in some more and look at the triangle mesh noise from laser scatter holes from Michelangelo’s drill artifacts from space carving 0.25 mm model photograph Ioannis Stamos – CSc Spring 2007 0:45

75 What do you think?

Download ppt "3D Computer Vision: CSc 83020."

Similar presentations

Ads by Google