1. Auto-Calibration of Multi-Projector Display Walls
Andrew Raij
UNC-Chapel Hill
University of Florida
Marc Pollefeys
UNC-Chapel Hill
12/1/2018
ICPR 2004

2. Multi-Projector Display
Projector misalignment
The tedious solution -- align by hand
Many automatic camera-based methods [Raskar Viz1999, Chen Vis2000, Yang Viz2001, Raij Procams2003, …]
ICPR 2004

3. Multi-Projector Display cont’d
Undefined display area
ICPR 2004

4. Related Work – Display Area
Rectangle on Screen
Require manual placement and / or interaction to achieve desired display area properties
Yang Vis2001, Rehg CARV2002, Raij Procams2003, Ashdown CVPR2004
Raskar CVPR2001, Raskar SIGGRAPH2003 – Requires tilt sensors and rigidly attached projector-camera pairs
Okatani ICCV2003 – Requires calibrated projectors
ICPR 2004

5. Main Contribution
We propose a fully automatic method for defining the display area of a planar multi-projector display
Single camera
No fiducials, interaction or additional sensors
Uncalibrated projectors
Note planar screen makes auto-calibration harder
ICPR 2004

6. Our Testbed: PixelFlex2
Planar, casually-aligned, multi-projector display system developed at UNC-Chapel Hill
8 projectors in 4x2 configuration, roughly 12' x 6' image
Calibration camera sees all projectors
Original system uses 4 fiducials to define display area
Note: without photometric correction
ICPR 2004

7. Outline of our Approach
Find camera-projector homographies
Projector calibration with planar auto-calibration (TriggsECCV98)
Reconstruct calibration camera, projectors and display plane.
Define world-aligned frame in plane and choose display area.
ICPR 2004

8. Step 1: Camera-Projector Homographies2 5 1 6 7 4 8 3
Features
Hcp
3.06
4.39e-02
-4.64e+02
-3.39e-02
2.76
-1.49e+03
1.69e-05
-2.17e-04
1.0
Proj ID
(X, Y) 1
(185.5, 113.5) 2
(347.5, 113.5) …
(…, …)
Camera
Proj ID
(X, Y) 1
(196.70, ) 2
(243.29, ) …
(…, …)
3.14
-1.06e-01
-1.79e+03
2.16e-03
2.83
-1.56e+03
4.55e-05
-2.34e-04
1.0
ICPR 2004

9. Step 2: Projector Auto-Calibration
Projector ≡ Camera
The image of the circular points must lie on the image of the absolute conic (IAC) in all views Ω∞ X π∞ πd xc ωc ωp
Hcpxc
Hcp C
xcTωcxc =
(Hcpxc)Tωp(Hcpxc)
0, p=1…n ω
K-TK-1 P
ICPR 2004

10. Projector Auto-Calibration cont’d
n projectors (n ≥ 3)
2n+2 constraints
n+5 unknowns
Camera is calibrated - ωc is known
Camera image of circ. points xc unknown
Projector - fpi, yp unknown
pixels are square
principal point same in all projectors, offset vertically
Note: Separate offset yields 2n+4 unknowns > 2n+2
Levenberg-Marquardt least squares minimization of
xcTωcxc =
(Hcpxc)Tωp(Hcpxc)
0, p=1…n
(0, yp) n
||xcTωcxc ||2 + ∑||(Hcpxc)Tωp(Hcpxc)||2
p=1
ICPR 2004

11. Initialization to Minimization
For each hypothesis on Kp and for each projector π1 π2
SVD-based Pose
Estimation for Planar
Scenes (TriggsECCV98)
Projector
Kp-1HcpKc
Camera C
xc1
xc2
||xcTωcxc ||2 +
∑||(Hcpxc)Tωp(Hcpxc)||2 n
p=1
ICPR 2004

12. Initialization (cont’d)
Clear minima produced by initialization algorithm
ICPR 2004

13. Estimated Projector Intrinsics
Geometric Configuration
Proj 1 2 3 4 5 6 7 8 YP
645.58
668.42
682.83
704.58
Intrinsics are consistent for different geometric configurations
ICPR 2004

14. Step 3: Reconstruction K1-1Hc1Kc SVD Kn-1HcnKcπ1 π2
K1-1Hc1Kc C P1
SVD π1
Kn-1HcnKc π2 C Pn
Camera-projector baseline is normalized to 1 so n reconstructions are in different frames, up to scale
Merge by normalizing by distance from camera to plane
ICPR 2004

15. Step 4: Display Area Selection
Display area is a world-aligned rectangle in the plane of desired aspect ratio
Vertical = Horizontal V Plane Normal
Assume camera x-axis is horizontal (i.e. allows tilting) and perform orthogonal projection on plane
Can choose rectangle of desired aspect ratio since reconstruction up to scale
ICPR 2004

16. Results
Camera, projector array, and plane reconstruction. 16:9 max inscribed area viewport shown in yellow.
Camera image with dotted lines extending towards vanishing points. Vanishing points found by projection of world direction basis into camera.
ICPR 2004

17. Conclusions Contributions
Automatic estimation of intrinsics of array of projectors projecting on a plane
Automatic estimation of projector extrinsics and display plane
Automatic selection of world-aligned viewport of proper aspect ratio
In sum, fully automatic calibration of a planar multi-projector /camera system without physical calibration objects and/or interaction.
ICPR 2004

18. Conclusions cont’d Discussion Future Work
Possible even though only scene observed by the camera and projectors is a plane!
“Metric" calibration is not highly accurate (intrinsics, etc.), but this is ok for the application. Perceptually, display area will appear to be a properly-aligned rectangle.
Future Work
Precise evaluation of calibration accuracy
Improve calibration with max likelihood estimation based on bundle-adjustment.
Calibrate for camera intrinsics, radial distortion in both camera and projectors.
ICPR 2004

19. Acknowledgements This work was partially supported by:
NSF Career award IIS
Department of Energy ASC VIEWS Program B519834
DARPA DARWARS program ONR N
The UNC PixelFlex Display Research Team
Henry Fuchs Herman Towles
Chris Ashworth Sundeep Tirumalareddy
ICPR 2004