1. Autonomous Vehicle Competition
Laksono Kurnianggoro
18 July 2014

2. Activities Last activities: Went to Seoul to test the AVC car.
Examined the image moments.
Modified the car tracking method.

3. IECON Registration

4. Image Moments
𝑚 𝑖𝑗 = 𝑥,𝑦 𝐼 𝑥,𝑦 𝑥 𝑖 𝑦 𝑗
𝑥 = 𝑚 10 𝑚 00 ; 𝑦 = 𝑚 01 𝑚 00

5. Problem with Car Detection Dataset
Some part of the dataset are background.
Affecting the tracking result.
Detected region
Detected region (HSV)
Backpojection image

6. Region used in histogram computation
Solution
Retrain the SVM using better dataset.
Need times to construct the new dataset.
OR:
Modify the tracking model.
Using a smaller window size for the histogram computation.
Detected region
Region used in histogram computation
result

7. Appendix

8. Part 1: Calibration of Camera and LRF
Relation between a point from camera’s and LRF’s point of view:
Where:
Φ is the rotation from camera to LRF.
Δ is the translation from camera to LRF.
𝑃 𝑐𝑎𝑚 location (in 3D space) is directly known,
However it is located at the calibration plane (checkerboard).
𝑃 𝐿𝑅𝐹 = Φ𝑃 𝑐𝑎𝑚 +Δ (1)
Calibration system’s setting

9. Point to Plane Constraint
Using camera calibration method, checkerboard’s normal and its distance to camera can be found.
The constraint:
In the camera coordinate system:
all the points (P) in checkerboard are constrained with its point to plane distance to the imaginary planar on the camera’s position.
Imaginary planar,
Parallel to the checker board
checkerboard
𝑁=− 𝑛 𝑑
− 𝑛 d
Camera’s position 𝑛
− 𝑛 .𝑃 𝑛 =𝑑
𝑁.𝑃= 𝑑 2 (2)

10. Formulation of the Calibration System
Main relation:
Simplification:
All laser points are located at Y=0  𝑃 𝐿𝑅𝐹 = 𝑋;𝑍;1
Formula (3) can be rewritten as:
Where:
Knowing N, d and 𝑃 𝐿𝑅𝐹 , H can be computed.
𝑁.𝑃= 𝑑 2 (2)
𝑃 𝑐𝑎𝑚 = Φ −1 (𝑃 𝐿𝑅𝐹 −Δ) (1)
𝑁. Φ −1 (𝑃 𝐿𝑅𝐹 −Δ)= 𝑑 2 (3)
𝑁.𝐻 𝑃 𝐿𝑅𝐹 = 𝑑 2 (4)
𝐻= Φ − Δ

11. Decomposing H into Rotation and Translation
Where 𝐻 𝑖 is the 𝑖 𝑡ℎ column of matrix H:
Φ= [ 𝐻 1 ,− 𝐻 1 × 𝐻 2 , 𝐻 2 ] ⊺
∆ = −[ 𝐻 1 ,− 𝐻 1 × 𝐻 2 , 𝐻 2 ] ⊺ 𝐻 3

12. Laser’s visualization
Result
Camera’s image
Laser’s visualization

13. Calibration in AVC’s Car

14. Conversion of 3D point to Image Point
Points3D, 𝐾, Kc
p.x=Points3d.x/Points3d.z
p.y=Points3d.y/Points3d.z
Tangential Distortion:
𝑎_1 𝑖 =2.0∗ 𝑝 𝑖 .𝑥+ 𝑝 𝑖 .𝑦
𝑎_2 𝑖 = 𝑟 𝑖 ∗ 𝑝 𝑖 .𝑥 2
𝑎_3 𝑖 = 𝑟 𝑖 ∗ 𝑝 𝑖 .𝑦 2
∆ 𝑥 𝑖 =𝑘𝑐 2 ∗ 𝑎_1 𝑖 +𝑘𝑐 3 ∗ 𝑎_2 𝑖
∆ 𝑦 𝑖 =𝑘𝑐 2 ∗ 𝑎_3 𝑖 +𝑘𝑐 3 ∗ 𝑎_1 𝑖
Distortion:
𝑟 𝑖 2 = 𝑝 𝑖 .𝑥 2 + 𝑝 𝑖 .𝑦 2 , 1<𝑖<𝑛𝑃𝑜𝑖𝑛𝑡𝑠
𝑟 𝑖 4 = 𝑟 𝑖 2 𝑟 𝑖 2
𝑟 𝑖 6 = 𝑟 𝑖 4 𝑟 𝑖 2
Radial Distortion:
𝑐𝑑𝑖𝑠𝑡 𝑖 =1.0+𝑘𝑐 0 ∗ 𝑟 𝑖 2 +𝑘𝑐 1 ∗ 𝑟 𝑖 4 +𝑘𝑐 4 ∗ 𝑟 𝑖 6
𝑝_𝑡𝑑 𝑖 .𝑥= 𝑝_𝑟𝑑 𝑖 .𝑥∗ ∆𝑥 𝑖
𝑝_𝑡𝑑 𝑖 .𝑦= 𝑝_𝑟𝑑 𝑖 .𝑦∗ ∆𝑦 𝑖
Skew:
𝛼=𝛾/𝑓𝑥
𝑝_𝑟𝑑 𝑖 = 𝑝 𝑖 ∗ 𝑐𝑑𝑖𝑠𝑡 𝑖
𝑝_𝑠 𝑖 .𝑥= 𝑝_𝑡𝑑 𝑖 .𝑥+𝛼∗ 𝑝 𝑡𝑑 𝑖 .𝑦
𝑝_𝑠 𝑖 .𝑦= 𝑝_𝑡𝑑 𝑖 .𝑦
𝑃𝑜𝑖𝑛𝑡𝑠 𝑜𝑛 𝑡ℎ𝑒 𝑖𝑚𝑎𝑔𝑒:
𝑝2𝑑.𝑥= 𝑝_𝑠 𝑖 .𝑥∗𝑓𝑥+𝑐𝑥
𝑝2𝑑.𝑦= 𝑝_𝑠 𝑖 .𝑦∗𝑓𝑦+𝑐𝑦
𝐾= 𝑓𝑥 𝛾 𝑐𝑥 0 𝑓𝑦 𝑐𝑦 0 0 1

15. Part 2: Registration Problem
In the dataset, the laser was not registered properly to the camera.
Solution: Using several sliding windows.
Red: sliding window boxes, cyan: detected car.

16. Part 4: Predicting the Bounding Box Position
Frame n
Frame n+1
When the result of car detection is available
= dx, dy
current
prev
When the result of car detection is available
prev
prediction
Frame n
Frame n+1

17. Part 5: Simple Clustering Method
Exploit the property of the LRF.
Invalid scan is detected as 0 meter.
If the different of the two consecutive scan data is more than a threshold, they belonged to different cluster.

18. Part 6: Tracking the Laser Data
Assumption: object in the new frame should not be moved far away compared to the previous frame.
Green line: previous position
Object candidate

19. Merging the LRF Tracking and Image Tracking
Calibration data
Cam
clustering
filtering
registration
candidates
Choose car clusters
Recognition (svm)
Choose the highest confidence region
LMS tracking: add tracked cluster
Detected cars region
Pick the cluster closest to the predictor
Image tracking
Pick the corresponding LRF cluster
A cluster
is picked?
Image tracking N
Use Image detection as reference Y
Update the predictor
Laser tracking

20. Output data from the tracking program
Adding the cluster length to output.
Visualization of the laser data
box_x
box_y
box_w
box_h
laser_x
laser_y
length

21. Mean-Shift Tracking model patch Histogram computation normalization
observation
Change the pixel value to corresponding histogram bin value
Observed image
Back-projection image
Mean-shift
1.0 30 50 60 90
Histogram example
Back-projection image example

22. Mean-Shift Finds the center of gravity from a given window.
Repeat until convergence.
Iteration #1
Iteration #2

23. Mean-Shift in Back-Projection Image
Mass center is computed using moment.
where
𝑥 = 𝑚 10 𝑚 00 ; 𝑦 = 𝑚 01 𝑚 00
𝑚 𝑖𝑗 = 𝑥,𝑦 𝐼 𝑥,𝑦 𝑥 𝑖 𝑦 𝑗
video
Illustration of the mean shift iteration on the back projection image sequences