1. 3D Single Image Scene Reconstruction For Video Surveillance Systems
Patrick Denis, Francisco Estrada and James Elder
Goal
Results
Compute 3D models of urban scenes from surveillance cameras.
Manhattan World
Manhattan World
Vanishing Points
Using homogeneous coordinates [4] for 3D points, we have:
Typical man-made urban structures are built as rectangles that lie at right angles to each other. Urban scenes that conform to an orthogonal three-dimensional grid are said to follow a “Manhattan World Assumption” [1].
Manhattan World rectangles must lie in one of the three canonical families of planes defined by the grid. Due to perspective projection, these rectangles appear in the image as general quadrilaterals.
A vanishing point is a point in the image where parallel lines intersect. The 3D orientation of these lines is given by the orientation of the line passing through the optical centre and the vanishing point [2].
Calculate the normal to the interpretation plane using the optical centre, and two homogeneous image points of a line segment (u1 & u2)
Calculate the plane  using the vanishing point vector , a homogeneous image point u1 and
Compute the intersection u3 between  and the line L2 defined by the join of u2 and the optical centre o using Plücker [4] notation
Calculate the angles of the triangle defined by the optical centre o, u1 and u3
Find  and , the distances from the optical centre to the 3D endpoints of the line segment.
Orientation of lines and Vanishing Points
Vanishing point
Introduction
We suggest that the strong constraints of an urban environment (the “Manhattan World Assumption”) and the internal parameters from a single camera are sufficient to infer enough of the three-dimensional context to produce a 3D rendering useful for visual surveillance. The 3D model is only reconstructed up to a projective scale.
The following steps are proposed to render an image of an urban environment as a three-dimensional model:
1) Extract the quadrilaterals from monocular video frames.
2) Infer the 3D orientation and location of the quadrilaterals.
3) Render the scene to the user.
At this time, quadrilaterals are identified interactively. We plan to automate this step in the future
Optical Centre
Health, Nursing & Environmental Studies building @ York University
Line through optical centre and vanishing point
Parallel Image lines
Figure 2. The line through the optical centre and the vanishing point has the same 3D orientation as the parallel lines identified as meeting at the vanishing point.
where
& 3D
Finding world coordinates of an image line
A line in the image can be generated by an infinite number of lines in space. The triangular plane defined by two rays that begin at the optical centre of the camera and continue through the two endpoints of the line in the image is known as the interpretation plane [3]. Any line segment in space that extends from one ray to the other in the interpretation plane will project to the same image line. Each of these lines is identified by its orientation and length.
Ross York University
Geometrical Representation to Find World Coordinates for an Image Line
Relationship between Image Line and 3D Lines
Conclusion
Image plane
|u1| 2 u1
|u1|
The effectiveness of urban visual surveillance may be enhanced if activities can be rendered within a three-dimensional context. Preliminary results show that semi-automatic inference of 3D models of urban buildings from monocular imagery is feasible, based on the Manhattan World assumption and the laws of perspective projection. Models are inferred up to a global scale factor. 2 g 1 L2 1 L1 o
|u2| u3 3
Interpretation plane
Image plane
|u2|
Image Line
Figure 4. Necessary steps to find the world coordinates of a line given the orientation and length.
Figure 1. User interface created to define the set of quadrilaterals following the Manhattan World Constraint.
Figure 1. User interface created to define the set of quadrilaterals following the Manhattan World Constraint.
Figure 3. A line in the image may project from any one of many lines in space. Each line in 3D space has a unique orientation  and length L.
Creating a Model
Creating a Model 3D
References
Texture Mapping
For purposes of texture mapping, we estimate a homography for each quadrilateral in the image that will map pixels from the image to a rectangle for the rendered environment.
To make a 3D model from a set of identified quadrilaterals in an image we must:
Identify the orientation of the Manhattan World relative to the camera optical centre with the use of Vanishing Points.
Find 3D world coordinates of an image line.
Find the remaining world coordinates for all defined quadrilaterals.
Extract the quadrilaterals in the image suitable for textures rendering.
To make a 3D model from a set of identified quadrilaterals in an image we must:
Identify the orientation of the Manhattan World relative to the camera optical centre with the use of Vanishing Points.
Find 3D world coordinates of an image line.
Find the remaining world coordinates for all defined quadrilaterals.
Extract the quadrilaterals in the image suitable for textures rendering.
[1] Coughlan, J. and Yuille A.L., “Manhattan World: Orientation and Outlier Detection by Bayesian Inference,” in Neural Computation, vol. 15, no. 5, pages , 2003
[2] Collins R.T. and Weiss R.S. “Vanishing Point Calculation as a Statistical Inference on the Unit Sphere,” In Proceedings of the Third International Conference on Computer Vision, pages , December 1990
[3] Stephen T. Barnard, “Interpreting Perspective Images,” in Artificial Intelligence, vol. 21, pages , 1982
[4] Hartley R. and Zisserman A., “Multiple View Geometry in Computer Vision,” Cambridge University Press, Second Edition, 2003
The 3D orientation of an image lines is determined using the vanishing point.
The length of a line is determined up to a global scale factor.
Figure 5. A homography maps an image quadrilateral to a rectangle.