1. DEPTH RANGE ACCURACY FOR PLENOPTIC CAMERAS
N. Monteiro, S. Marto, J. Barreto and J. Gaspar
ISR LISBOA / ISR COIMBRA / LARSyS
Abstract
Plenoptic cameras capture the directional information of the light distribution from a scene. This is accomplished by positioning a microlens array between the main lens and the sensor. This configuration obtains multiple projections for a scene point, which allows to retrieve the point's depth on a single exposure. In recent years, several studies recover depth and shape from the lightfield data using several cues. These studies focus on non-metric reconstruction or relative depth. In this work, we formalize a projection model and define a metric reconstruction methodology for a calibrated standard plenoptic camera to evaluate the depth estimation accuracy for these cameras. The metric reconstruction methodology is applied to a public dataset containing calibration grids, at different depths. The results indicate that these cameras are capable of reconstructing depth with high metric accuracy for points near the camera. The accuracy of the depth reconstruction is improved by imposing consistency on input data.
Projection and Reconstruction
Let us consider the lightfield parameterized by one direction (u,v) and one position (s,t). Let us also consider the intrinsic matrix H that allows to map the lightfield in the image sensor (i,j,k,l) to the lightifled in the object space (s,t,u,v). Considering this mapping and the relation of (s,t,u,v) and a point in the object space:
The projection is given by:
Considering each position (s,t) as a pinhole, we can define the reconstruction as a multi-view stereo problem. Thus, considering that we have z correspondences
SUBTITLE: Estimated (red) and ground truth (blue) calibration grid points obtained from the calibration datasets A, B, and C provided by Dansereau et al. [1] for different poses of the calibration pattern.
Depth Estimation Experiments
The reconstruction methodology is applied to calibration datasets were the calibration grids are placed at a known depth. The reconstruction is complemented with the reconstruction of randomly selected scene points obtained after projection.
The findings suggest a depth range between 0 and 1.5 m, which limits the reconstruction to points near the camera.
SUBTITLE: Reconstructed depth for dataset A (green), B (blue), and C (cyan) superimposed by the estimates obtained for the calibration grid points of poses 1, 5, and 9 of each dataset (red dots).
References
[1] Dansereau, Donald G., Oscar Pizarro, and Stefan B. Williams. "Decoding, calibration and rectification for lenselet-based plenoptic cameras." Proceedings of the IEEE conference on computer vision and pattern recognition