Presentation is loading. Please wait.

Presentation is loading. Please wait.

Kawada Industries Inc. has introduced the HRP-2P for Robodex 2002

Similar presentations

Presentation on theme: "Kawada Industries Inc. has introduced the HRP-2P for Robodex 2002"— Presentation transcript:

1 ROBOT VISION Lesson 7: State of the Art in 3D Reconstruction Matthias Rüther
Kawada Industries Inc. has introduced the HRP-2P for Robodex 2002.  This humanoid appears to be very impressive. It is 154 cm (60") tall, weighs 58kg (127 lbs) and has 30 DOF. Here is a news release. Notice the LACK of a battery pack.  Here is a new story about HRP2. 

2 Overview Structured light Plane Sweep Stereo
Shape from X techniques Structured light Plane Sweep Stereo Global optimization methods Shape from (de)focus Specularities Shading, Photometric Stereo Alternative methods

3 Structured Light Methods
In principle same as multi-view/stereo Project artificial pattern on the object Pattern alleviates the correspondence problem Variants: Laser Pattern (point, line) Structured projector pattern (several lines, pattern sequence) Random projector pattern

4 Structured Light Range Finder
1. Sender (projects plane) 2. Receiver (CCD Camera) Geometry Z- direction X- direction Sensor image

5 1 plane -> 1 object profile
To get a 3D profile: Move the object Scanning Unit for projected plane Move the Sensor Object motion by conveyor band: => synchronization: measure distance along conveyor => y-accuracy determined by distance measurement Scanning Units (e.g.: rotating mirror) are rare (accurate measurement of mirror motion is hard, small inaccuracy there -> large inaccuracy in geometry Move the sensor: e.g. railways: sensor in wagon coupled to speed measurement


7 Commercially Available

8 CAESARTM Civilian American and European Surface Anthropometry Resource Project—CAESARTM 2400 Male/Female Americans 2000 Male/Female Europeans

9 Problems Occlusions Sharpness and Contrast Speckle noise

10 Gleichzeitige Projektion mehrerer Lichtschnitte
Anstatt einer Lichtebene werden mehrere Lichtebenen auf das Objekt projeziert, um die Anzahl der aufzunehmenden Bilder zu reduzieren. Entfernungsberechnung: wie mit einer Lichtebene, jedoch muß jeder Lichtstreifen im Bild eindeutig identifizierbar sein. Problem: Aufgrund von Verdeckungen sind einzelne Streifen teilweise oder gar nicht im Kamerabild sichtbar -> keine eindeutige Identifikation der Lichtstreifen Anwendung: Glattheitsüberprüfung bei planaren Oberflächen ohne Tiefenwertberechnung. 1) peak at position xo 2) left side und right side of the peal using a threshold Vt 3) center points

11 Pattern projection Camera: IMAG CCD, Res:750x590, f:16 mm
Projected light stripes Camera: IMAG CCD, Res:750x590, f:16 mm Projector: Liquid Crystal Display (LCD 640), f: 200mm, Distance to object plane: 120cm Range Image

12 Line projector (z.b: LCD-640)
Lamp Lens system LCD - Shutter Pattern structure Line projector (z.b: LCD-640) Focusing lens (e.g.: 150mm) Example

13 Tiefenberechnung für Streifenprojektor
Durch die zeitliche Abfolge der Aufnahmen wird es ermöglicht, daß jeder Lichtstreifen im Kamerabild identifiziert wird. Verschiedene Lichtstreifen sind notwendig, um für jeden Bildpunkt einen zugehörigen Lichtstreifen festzustellen zu können. 1) Unterschiedlich breite Lichtstreifen werden zeitlich aufeinanderfolgend in die Szene projiziert und von der Kamera aufgenommen. 2) Für jede Aufnahme wird für jeden Bildpunkt festgestellt, ob dieser beleuchtet wird oder nicht. 3) Diese Information wird für jeden Bildpunkt und für jede Aufnahme im sog. Bit-Plane Stack abgespeichert. 4) Findet man im Bit-Plane Stack für einen Bildpunkt die Information, daß er bei den Aufnahmen z.B. hell, dunkel, dunkel, hell war (Code 1001), dann folgt daraus, daß dieser Bildpunkt vom vierten Lichtstreifen beleuchtet wird. 5) eindeutige Zuordnung „Lichtstreifen – Bildpunkt“ möglich

14 Coded Light + Phase Shift
Binary code is limited to pixel accuracy (at most). Increase accuracy by projecting sine wave and measuring phase shift between projected and captured pattern.

15 Joaquim Salvi, Pattern codification strategies in structured light systems

16 Random Texture Projection

17 Moiré Range Finder 1. Sender (Projektor mit Linien)
Project line structure, observe line structure through a grid 1. Sender (Projektor mit Linien) 2. Receiver (CCD Camera with line filter) Problem: identification of line ordering possible but hard, unsharp lines => inaccurate results Moiré Pattern Moiré Image

18 Moiré Range Finder Y. Suenaga, 3D Measurements for Computer Animation

19 Plane Sweep Stereo Sweep family of planes through volume
 projective re-sampling of (X,Y,Z) input image each plane defines an image  composite homography Richard Szeliski, IBMR 1998

20 Plane Sweep Stereo For each depth plane
compute composite (mosaic) image — mean compute error image — variance convert to confidence and aggregate spatially Select winning depth at each pixel Richard Szeliski, IBMR 1998

21 Voxel Coloring

22 Voxel Coloring / Space Carving
S={} /* initial set of colored voxels is empty for i = 1 to r do /* traverse each of r layers foreach V in the ith layer of voxels do project V into all images where V is visible if sufficient correlation of the pixel colors then add V to S Photorealistic Scene Reconstruction by Voxel Coloring S. M. Seitz and C. R. Dyer, Proc. Computer Vision and Pattern Recognition Conf., 1997,

23 Shape From Focus Recover shape of surfaces from limitied depth of view. Requires visibly rough surfaces Typical application: optical microscopy

24 Shape From Focus Visibly Rough Surfaces
Optical roughness: the smallest spatial variations are much larger than the wavelength of incident electromagnetic wave. Visible roughness: smallest spatial variations are comparable to viewing area of discrete elements (pixels). Magnification: Multi-facet level: w1 >> wf, smooth texture Facet level: w2 ~= wf, rough texture

25 Shape From Focus Focused / Defocused images Focused:
Defocused: object point is mapped to spot with radius => defocusing is equivalent to convolution with low pass kernel (pillbox function)

26 Shape From Focus Changing Focus
Displacing the sensor: changes sharp region, magnification and brightness Moving the lens: changes sharp region, magnification and brightness Moving the object: changes sharp region only => Object is moved in front of static camera

27 Shape From Focus Overview:
At facet level magnification, rough surfaces give texture-rich images A defocused image is equivalent to a low-passed image As S moves towards focused plane, its focus increases. When S is best focused, Challenges: How to measure focus? How to find best focus from finite number of measurements?

28 Shape From Focus Focus measure operator Purpose: Possible solution:
respond to high frequency variations in image intensity within a small image area produce maximum response when image area is perfectly focused Possible solution: determine high frequency content using Fourier transform (slow) Alternative: Laplacian operator (problem with elimination) Modified Laplacian

29 Shape From Focus INxM … local intensity function (image window)
Sum Modified Laplacian Tenengrad Focus Measure Alternatives: variance of intensities, variance of gradients INxM … local intensity function (image window)

30 Shape From Focus Example Infinite DOF DEM

31 Shape From Focus Sampling the focus measure function
Consider a single image point (x,y) Focus measure F is function of depth d: F(d) Goal find Fpeak from finite number of samples F1…F8

32 Shape from focus Sampling the focus measure function
Possibility1: find highest discrete sample

33 Shape from focus Sampling the focus measure function
Possibility2: Gaussian interpolation Fit Gauss function to three strongest samples

34 Shape from Specularity
Suitable for highly reflective Surfaces Specular Reflection map of a single point source forms a sharp peak (Specular model, Phong model) Reflectance map

35 Shape from Specularity
Principle: If a reflection is seen by the camera and the position of the point source is known, the surface normal can be determined. => use several point sources with known position: structured highlight inspection Panorama mirror

36 Shape From Shadow Also: Shape from Darkness
Reconstruct Surface Topography from self-occlusion E.g. Building reconstruction in SAR images, terrain reconstruction in remote sensing

37 Shape From Shadow A static camera C observes a scene.
Light source L travels over the scene x, position of L is given by angle . L and C are an infinite distance away (orthographic projection). Shadowgram: binary function f(x, ), stating whether scene point x was shadowed at light position .

38 Photometric Stereo Multiple images, static camera, different illumination directions At least three images Known illumination direction Known reflection model (Lambert) Object may be textured

L = E (a COS q + b COS a) n albedo Diffuse albedo Specular a=0.3, b=0.7, n= a=0.7, b=0.3, n=0.5

40 Photometric Stereo Albedo (reflectivity) Reflection model
3 unknowns per pixel at least three illumination directions

41 Photometric Stereo surface normal albedo light direction

42 Photometric Stereo: Example
Input images From Forsyth & Ponce Recovered normals Recovered albedo

43 Range Finder principles:
Runtime Range Finder Triangulation Range Finder - Sender and receiver with known position, triangulation similar to stereo principle Optical Range Finder Ultrasound Range Finder e.g.: Spot Projectors Moiré Range Finder Structured Light Range Finder Pattern (stripe) projection 1 image 2 dim + geometry => 3 dim Depth information Receiver/Sender position

44 Runtime Range Finder Determine sensor-object distance by measuring radiation runtime: 1. Sender (coherent light) 2. Scanning Unit 3. Receiver (light detector) 4. Phase detector Alt. Method: send light pulses => LIDAR (Light Radar), defense industry Problem: generating pulses, measuring runtime (both very short) Die Signalphaserelativ zur reference phase ist proportional zur Laufzeit. Die Signalamplitude ist proportional zur zur Oberflächenreflektion. => Lichtintensität und Range Data

45 Ultrasound Range Finder
Used in commercial cameras (Autofocus Spot) Advantages Independent of sorrounding light, Slow speed of ray Disadvantages coarse resolution, Bad accuracy, Pointwise, scanner necessary Multiple Reflection/Echoes Applications: Obstacle Detection: e.g.: “Car parking radar” Level Measurement, silos, tanks, … Underwater ranging (sonar), ... Typ. specifications: Range: 5cm to 1-5m Accuracy: +-3mm

Download ppt "Kawada Industries Inc. has introduced the HRP-2P for Robodex 2002"

Similar presentations

Ads by Google