1. Omnidirectional camera calibration
Ph.D. student
Chang-Ryeol Lee
November 26, 2013

2. Contents Camera model What is camera calibration? Camera calibration
Tsai’s and Zhang’s methods
Omnidirectional camera
Calibration: A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion

3. Camera model
* Prof. Kuk-Jin Yoon’s slides on High-level Image Understanding and Processing

4. What is calibration?
* Prof. Kuk-Jin Yoon’s slides on High-level Image Understanding and Processing

5. Camera model
* Prof. Kuk-Jin Yoon’s slides on High-level Image Understanding and Processing

6. What is camera calibration?

7. What is camera calibration?

8. Camera calibration
We know positions of pattern corners with respect to world and image coordinate
3D corners
2D corners x y z
{World}
Grid size:3cm
{Image}

9. Camera calibration Camera parameters Intrinsic parameter K
Focal length, skew, principal point
Extrinsic parameter P
Rotation, translation

10. Tsai’s and Zhang’s methods
Retrieve not only the intrinsic but also extrinsic parameters
Set their Z-coordinate to 0
because the 3D points on the object are coplanar.
A homography between m and M
m ≅𝐊 𝐑 T | −𝐑 T t M=𝐊 r 1 r 2 r 3 t ′ X Y 0 1 =𝐊 r 1 r 2 t ′ X Y 1
𝐇= h 1 h 2 h 3 =λ𝐊 r 1 r 2 t ′

11. Tsai’s and Zhang’s methods
Solving for Homography

12. Tsai’s and Zhang’s methods
Any rotation matrix is an orthonormal matrix
Define the symmetric matrix B as
- 6 unknown variable
r 1 r 2 t ′ = 𝐊 −1 h 1 h 2 h 3
r 1 𝑇 r 2 = ( 𝐊 −1 h 1 ) 𝑇 𝐊 −1 h 2 = h 2 𝑇 𝐊 −𝑇 𝐊 −1 h 2 =0
r 1 𝑇 r 1 = r 2 𝑇 r 2 = ( 𝐊 −1 h 1 ) 𝑇 𝐊 −1 h 1 = h 1 𝑇 𝐊 −𝑇 𝐊 −1 h 1 = h 2 𝑇 𝐊 −𝑇 𝐊 −1 h 2
𝐁= 𝐊 −𝑇 𝐊 −1 = 𝐵 11 𝐵 12 𝐵 13 𝐵 12 𝐵 22 𝐵 23 𝐵 13 𝐵 23 𝐵 = 0

13. Tsai’s and Zhang’s methods
One image offers two linear equations in elements of B
We can get intrinsic parameter through 3 or more images
because unknown variable( 𝐵 11 ⋯ 𝐵 33 ) is 6.
h 1 𝑇 𝐁 h 1 = h 2 𝑇 𝐁 h 2 , h 2 𝑇 𝐁 h 2 =0

14. Tsai’s and Zhang’s methods
Derive equations for the internal parameters of K
𝐵 11 𝐵 12 𝐵 13 𝐵 12 𝐵 22 𝐵 23 𝐵 13 𝐵 23 𝐵 = 𝛼 𝑥 𝑠 𝑝 𝑥 0 𝛼 𝑦 𝑝 𝑦 −𝑇 𝛼 𝑥 𝑠 𝑝 𝑥 0 𝛼 𝑦 𝑝 𝑦 −1
𝐁 = 𝐊 −𝑇 𝐊 −1
𝑢 𝑦 = 𝐵 12 𝐵 13 − 𝐵 13 𝐵 23 𝐵 11 𝐵 22 − 𝐵 12 2
λ= 𝐵 33 − 𝐵 𝑢 𝑦 ( 𝐵 12 𝐵 13 − 𝐵 11 𝐵 23 ) 𝐵 11
𝛼 𝑦 = λ 𝐵 11 𝐵 11 𝐵 22 − 𝐵 12 2
𝛼 𝑥 = λ 𝐵 11
𝑠=− 𝐵 12 𝛼 𝑥 2 𝛼 𝑦 λ
𝑢 𝑥 = 𝑠 𝑢 𝑦 𝛼 − 𝐵 13 𝛼 2 λ

15. Tsai’s and Zhang’s methods
After intrinsic parameters, the extrinsics readily follow
r 1 r 2 t ′ = 𝐊 −1 h 1 h 2 h 3
r 1 = μ 𝐊 −1 h 1
r 2 = μ 𝐊 −1 h 2
t = μ 𝐊 −1 h 3
μ = 1 𝐊 −1 h 1
r 3 = r 1 × r 2

16. Omnidirectional camera
Dioptric (fish-eye lens)
Catadioptric
(camera with mirror)

17. Omnidirectional camera
Camera model
Catadioptric case
Dioptric case

18. A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion
Don’t care whether catadioptric or dordioptric
General method even NASA use this currently
Key idea
Camera modeling by Taylor expansion

19. A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion
No focal length, skew, principal point
Calibration parameters
Taylor expansion coefficients a
Rotation R
Translation t

20. A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion
Set their Z-coordinate to 0 by planarity of pattern

21. A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion
Cross product of a corner point
make three equations

22. A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion
The 3rd equation can be linearly expressed on unknowns
We know
Homogeneous linear least square problem is solved by SVD

23. A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion
𝑟 3 is uniquely determined by 𝑟 1 , 𝑟 2
Extrinsic parameter is determined except for 𝑡 3
The 1st, 2nd equations are used to determine
, where

24. A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion
Equations stacked with K observations of pattern is expressed linearly
where
Unknown parameter 𝑡 3 ,𝐚 are estimated
1st
Kth

25. Question or Comments ?