Presentation is loading. Please wait.

Presentation is loading. Please wait.

Robot Vision SS 2009 Matthias Rüther 1 ROBOT VISION Lesson 7: State of the Art in 3D Reconstruction Matthias Rüther.

Similar presentations


Presentation on theme: "Robot Vision SS 2009 Matthias Rüther 1 ROBOT VISION Lesson 7: State of the Art in 3D Reconstruction Matthias Rüther."— Presentation transcript:

1 Robot Vision SS 2009 Matthias Rüther 1 ROBOT VISION Lesson 7: State of the Art in 3D Reconstruction Matthias Rüther

2 Robot Vision SS 2009 Matthias Rüther 2 Overview Shape from X techniques –Structured light –Plane Sweep Stereo –Global optimization methods –Shape from (de)focus –Specularities –Shading, Photometric Stereo –Alternative methods

3 Robot Vision SS 2009 Matthias Rüther 3 Structured Light Methods In principle same as multi-view/stereo Project artificial pattern on the object Pattern alleviates the correspondence problem Variants: –Laser Pattern (point, line) –Structured projector pattern (several lines, pattern sequence) –Random projector pattern

4 Robot Vision SS 2009 Matthias Rüther 4 Structured Light Range Finder 1. Sender (projects plane) 2. Receiver (CCD Camera) X- directionGeometry Z- direction Sensor image

5 Robot Vision SS 2009 Matthias Rüther 5 1 plane -> 1 object profile Object motion by conveyor band: => synchronization: measure distance along conveyor => y-accuracy determined by distance measurement Scanning Units (e.g.: rotating mirror) are rare (accurate measurement of mirror motion is hard, small inaccuracy there -> large inaccuracy in geometry Move the sensor: e.g. railways: sensor in wagon coupled to speed measurement To get a 3D profile: Move the object Scanning Unit for projected plane Move the Sensor

6 Robot Vision SS 2009 Matthias Rüther 6

7 7 Commercially Available

8 Robot Vision SS 2009 Matthias Rüther 8 CAESAR TM Civilian American and European Surface Anthropometry Resource ProjectCAESAR TM –2400 Male/Female Americans –2000 Male/Female Europeans

9 Robot Vision SS 2009 Matthias Rüther 9 Problems Occlusions Sharpness and Contrast Speckle noise

10 Robot Vision SS 2009 Matthias Rüther 10 Gleichzeitige Projektion mehrerer Lichtschnitte Anstatt einer Lichtebene werden mehrere Lichtebenen auf das Objekt projeziert, um die Anzahl der aufzunehmenden Bilder zu reduzieren. Entfernungsberechnung: wie mit einer Lichtebene, jedoch muß jeder Lichtstreifen im Bild eindeutig identifizierbar sein. Problem: Aufgrund von Verdeckungen sind einzelne Streifen teilweise oder gar nicht im Kamerabild sichtbar -> keine eindeutige Identifikation der Lichtstreifen Anwendung: Glattheitsüberprüfung bei planaren Oberflächen ohne Tiefenwertberechnung.

11 Robot Vision SS 2009 Matthias Rüther 11 Pattern projection Camera Camera: IMAG CCD, Res:750x590, f:16 mm Projector Projector: Liquid Crystal Display (LCD 640), f: 200mm, Distance to object plane: 120cm Projected light stripes Range Image

12 Robot Vision SS 2009 Matthias Rüther 12 Projector Lamp Lens system LCD - Shutter Pattern structure Example Focusing lens (e.g.: 150mm) Line projector (z.b: LCD-640)

13 Robot Vision SS 2009 Matthias Rüther 13 Tiefenberechnung für Streifenprojektor 1) Unterschiedlich breite Lichtstreifen werden zeitlich aufeinanderfolgend in die Szene projiziert und von der Kamera aufgenommen. 2) Für jede Aufnahme wird für jeden Bildpunkt festgestellt, ob dieser beleuchtet wird oder nicht. 3) Diese Information wird für jeden Bildpunkt und für jede Aufnahme im sog. Bit-Plane Stack abgespeichert. Verschiedene Lichtstreifen sind notwendig, um für jeden Bildpunkt einen zugehörigen Lichtstreifen festzustellen zu können. Durch die zeitliche Abfolge der Aufnahmen wird es ermöglicht, daß jeder Lichtstreifen im Kamerabild identifiziert wird. 4) Findet man im Bit-Plane Stack für einen Bildpunkt die Information, daß er bei den Aufnahmen z.B. hell, dunkel, dunkel, hell war (Code 1001), dann folgt daraus, daß dieser Bildpunkt vom vierten Lichtstreifen beleuchtet wird. 5) eindeutige Zuordnung Lichtstreifen – Bildpunkt möglich

14 Robot Vision SS 2009 Matthias Rüther 14 Coded Light + Phase Shift Binary code is limited to pixel accuracy (at most). Increase accuracy by projecting sine wave and measuring phase shift between projected and captured pattern.

15 Robot Vision SS 2009 Matthias Rüther 15 Joaquim Salvi, Pattern codification strategies in structured light systems

16 Robot Vision SS 2009 Matthias Rüther 16 Random Texture Projection

17 Robot Vision SS 2009 Matthias Rüther 17 Moiré Range Finder Project line structure, observe line structure through a grid 1. Sender (Projektor mit Linien) 2. Receiver (CCD Camera with line filter) Problem: identification of line ordering possible but hard, unsharp lines => inaccurate results Moiré ImageMoiré Pattern

18 Robot Vision SS 2009 Matthias Rüther 18 Moiré Range Finder Y. Suenaga, 3D Measurements for Computer Animation

19 Robot Vision SS 2009 Matthias Rüther 19 Plane Sweep Stereo Sweep family of planes through volume –each plane defines an image composite homography input image projective re-sampling of (X,Y,Z) Richard Szeliski, IBMR 1998

20 Robot Vision SS 2009 Matthias Rüther 20 Plane Sweep Stereo For each depth plane –compute composite (mosaic) image mean –compute error image variance –convert to confidence and aggregate spatially Select winning depth at each pixel Richard Szeliski, IBMR 1998

21 Robot Vision SS 2009 Matthias Rüther 21 Voxel Coloring

22 Robot Vision SS 2009 Matthias Rüther 22 Voxel Coloring / Space Carving S={} /* initial set of colored voxels is empty for i = 1 to r do /* traverse each of r layers foreach V in the ith layer of voxels do project V into all images where V is visible if sufficient correlation of the pixel colors then add V to S Photorealistic Scene Reconstruction by Voxel Coloring Photorealistic Scene Reconstruction by Voxel Coloring S. M. Seitz and C. R. Dyer, Proc. Computer Vision and Pattern Recognition Conf., 1997, 1067-1073

23 Robot Vision SS 2009 Matthias Rüther 23 Shape From Focus Recover shape of surfaces from limitied depth of view. –Requires visibly rough surfaces –Typical application: optical microscopy

24 Robot Vision SS 2009 Matthias Rüther 24 Shape From Focus Visibly Rough Surfaces –Optical roughness: the smallest spatial variations are much larger than the wavelength of incident electromagnetic wave. –Visible roughness: smallest spatial variations are comparable to viewing area of discrete elements (pixels). –Magnification: Multi-facet level: w 1 >> w f, smooth texture Facet level: w 2 ~= w f, rough texture

25 Robot Vision SS 2009 Matthias Rüther 25 Shape From Focus Focused / Defocused images –Focused: –Defocused: object point is mapped to spot with radius => defocusing is equivalent to convolution with low pass kernel (pillbox function)

26 Robot Vision SS 2009 Matthias Rüther 26 Shape From Focus Changing Focus –Displacing the sensor: changes sharp region, magnification and brightness –Moving the lens: changes sharp region, magnification and brightness –Moving the object: changes sharp region only => Object is moved in front of static camera

27 Robot Vision SS 2009 Matthias Rüther 27 Shape From Focus Overview: –At facet level magnification, rough surfaces give texture-rich images –A defocused image is equivalent to a low-passed image –As S moves towards focused plane, its focus increases. When S is best focused, –Challenges: How to measure focus? How to find best focus from finite number of measurements?

28 Robot Vision SS 2009 Matthias Rüther 28 Shape From Focus Focus measure operator –Purpose: respond to high frequency variations in image intensity within a small image area produce maximum response when image area is perfectly focused –Possible solution: determine high frequency content using Fourier transform (slow) –Alternative: Laplacian operator (problem with elimination) Modified Laplacian

29 Robot Vision SS 2009 Matthias Rüther 29 Shape From Focus Sum Modified Laplacian Tenengrad Focus Measure Alternatives: variance of intensities, variance of gradients I NxM … local intensity function (image window)

30 Robot Vision SS 2009 Matthias Rüther 30 Shape From Focus Example Infinite DOFDEM

31 Robot Vision SS 2009 Matthias Rüther 31 Shape From Focus Sampling the focus measure function –Consider a single image point (x,y) –Focus measure F is function of depth d: F(d) –Goal find F peak from finite number of samples F 1 …F 8

32 Robot Vision SS 2009 Matthias Rüther 32 Shape from focus Sampling the focus measure function –Possibility1: find highest discrete sample

33 Robot Vision SS 2009 Matthias Rüther 33 Shape from focus Sampling the focus measure function –Possibility2: Gaussian interpolation Fit Gauss function to three strongest samples

34 Robot Vision SS 2009 Matthias Rüther 34 Shape from Specularity Suitable for highly reflective Surfaces Specular Reflection map of a single point source forms a sharp peak (Specular model, Phong model)

35 Robot Vision SS 2009 Matthias Rüther 35 Shape from Specularity Principle: –If a reflection is seen by the camera and the position of the point source is known, the surface normal can be determined. –=> use several point sources with known position: structured highlight inspection

36 Robot Vision SS 2009 Matthias Rüther 36 Shape From Shadow Also: Shape from Darkness Reconstruct Surface Topography from self-occlusion E.g. Building reconstruction in SAR images, terrain reconstruction in remote sensing

37 Robot Vision SS 2009 Matthias Rüther 37 Shape From Shadow A static camera C observes a scene. Light source L travels over the scene x, position of L is given by angle. L and C are an infinite distance away (orthographic projection). Shadowgram: binary function f(x, ), stating whether scene point x was shadowed at light position.

38 Robot Vision SS 2009 Matthias Rüther 38 Photometric Stereo Multiple images, static camera, different illumination directions At least three images Known illumination direction Known reflection model (Lambert) Object may be textured

39 Robot Vision SS 2009 Matthias Rüther 39 REFLECTANCE MODELS albedo Diffuse albedo Specular albedo PHONG MODEL L = E (a COS b COS ) n a=0.3, b=0.7, n=2 a=0.7, b=0.3, n=0.5 LAMBERTIAN MODEL L = E COS

40 Robot Vision SS 2009 Matthias Rüther 40 Photometric Stereo Reflection model 3 unknowns per pixel Albedo (reflectivity) at least three illumination directions

41 Robot Vision SS 2009 Matthias Rüther 41 Photometric Stereo surface normal albedo light direction

42 Robot Vision SS 2009 Matthias Rüther 42 Photometric Stereo: Example From Forsyth & Ponce Input images Recovered albedo Recovered normals

43 Robot Vision SS 2009 Matthias Rüther 43 Range Finder Range Finder principles: Runtime Range Finder Triangulation Range Finder - Sender and receiver with known position, triangulation similar to stereo principle 1 image Receiver/Sender position Depth information 2 dim + geometry => 3 dim Optical Range Finder Ultrasound Range Finder e.g.: Spot Projectors Moiré Range Finder Structured Light Range Finder Pattern (stripe) projection

44 Robot Vision SS 2009 Matthias Rüther 44 Runtime Range Finder Determine sensor-object distance by measuring radiation runtime: 1. Sender (coherent light) 2. Scanning Unit 3. Receiver (light detector) 4. Phase detector Alt. Method: send light pulses => LIDAR (Light Radar), defense industry Problem: generating pulses, measuring runtime (both very short)

45 Robot Vision SS 2009 Matthias Rüther 45 Ultrasound Range Finder Used in commercial cameras (Autofocus Spot) Advantages Independent of sorrounding light, Slow speed of ray Applications: Obstacle Detection: e.g.: Car parking radar Level Measurement, silos, tanks, … Underwater ranging (sonar),... Typ. specifications: Range: 5cm to 1-5m Accuracy: +-3mm Disadvantages coarse resolution, Bad accuracy, Pointwise, scanner necessary Multiple Reflection/Echoes


Download ppt "Robot Vision SS 2009 Matthias Rüther 1 ROBOT VISION Lesson 7: State of the Art in 3D Reconstruction Matthias Rüther."

Similar presentations


Ads by Google