1. V4 – Video Tracker for Extremely Hard Night Conditions
December 2011 Omer Cohen Ophir Gozes Davidov Gavriel

2. Table Of Contents The challenge Demonstration videos Usual solutions
Our solution
Basic Assumptions for Tracking
Block Diagram
Performance examination
Conclusion

3. The Challenge Pointing and tracking a maneuvering target.
No prior knowledge of the target’s shape.
Dealing with hiding objects – Position estimation
Dealing with very low SNR videos.
Display velocity and acceleration of the tracked target.

4. Demonstration Videos – FLIR Human
FLIR video of a human crawling at night.
Very low SNR.
Watch video

5. Demonstration Videos – Maneuvering Super-Hornet
Night video of a maneuvering Super-Hornet.
Very low SNR.
Watch video

6. Demonstration Videos – Severely Maneuvering Particle
A Severely maneuvering particle.
High speed. Accelerating.
Hiding object
Low SNR.
Watch video

7. Combined Scenarios
Combining all 3 Scenarios into one movie, one after another.
The algorithms may cope with the scenario changes and identify the target automatically.

8. Usual Solutions Center of mass / Image Processing
Finding the maximal value at the frame:
Wrong, if working with a low SNR video is required.
A pre-processing filtering is required.
Correlation – with a prior known object
Limited for dealing with certain
known objects.

9. Our Solution
Combines 2 methods of objects tracking. Each of them with his own advantages.
Correlation of 2-4 following frames, based on (Quantitative Comparison of Algorithms for Tracking Single Fluorescent Particles) Cheezum,Walker,2001
Ideal for very low SNR videos. Limited for objects at medium velocity.
Image Processing Peak Search
Deals with high velocities at higher SNR.

10. Correlating Following Frames
We’ve based our solution on a method of Correlating Following Frames
Based on “name of an article”
Frame x+1
Frame x X

11. Correlating Following Frames (2)
We’ve implemented a 4-frame correlator
Better sensitivity (ADD THE CALCULATION)
Less fitted for tracking accelerating targets
Frame x+3
Frame x+2
Frame x+1 X
Frame x X X

12. 4 Frame Correlator Demonstration
The 2 frames correlation matrix before and after filtering. The peak cannot be detected.
Example: a rectangle at a noisy frame. Very low SNR. The object in hardly noticed.
The 4 frames correlation matrix before and after filtering. The peak is clearly detected.

13. The Correlation Matrix
Out of the correlation matrix one can calculate the object’s velocity in each axis.
However, the correlation matrix doesn’t give object’s location.
Center Of Frame
Frame x+1
Frame x X
y Velocity
X Velocity
Max Correlation Point

14. Tracking Controller - Correlation
In order to determine object’s position by the correlation method, we need to produce some spatial information about the object:
Minimize the frame into 4 sub-frames.
Correlate each 4 sub-frames with his following sub-frames.
Choose “best” correlation matrix.
Begin a new iteration.
Frame x+1
Frame x Up
Left
Right X
Down

15. Low-Pass: Convolution with a nXn square
Image Processing
Because of the noisy frames we perform a simple image processing: filtering with a Low-Pass filter.
Then we can clearly find maximum values.
Low-Pass: Convolution with a nXn square

16. Tracking Controller
The purpose is to combine our tracking methods (correlation of previous frames & center of mass) into one efficient algorithm, bringing each method’s advantages.
New frame arrived
if( first frame in the movie
|| certainty (last frame) < threshold
|| velocity (last frame) > threshold
Determine object’s position by IP
Determine object’s position by correlation result
Decide if object is present in the correlation frame
Certainty > threshold -> Object estimated position
Kalman Filter
Correlate using new frame

17. Basic Assumptions for tracking
IP Algorithm Thr.
Correlation Algorithm Thr.
Parameter
---
0.5 X Object’s width (pixels) / Frame
Velocity 1
0.3
SNR =
for Wobject ~ 0.05 X Wframe
Fairly Symmetric
wObject< wFrame
Object’s Shape

18. Dealing With Hiding Targets
In order to deal with hiding targets, we should be able to filter frames in which the object is absent.
We use a 4 dimension Kalman filter (Xx,Xy,Vx,Vy) to filter the samples.
The certainty generated by the correlation algorithm is used as the samples noise.
Watch video

19. Performance Examination
Algorithm result for combined videos: Watch video Algorithm Result for combined videos (2): Algorithm result - hiding objects:

20. Low Snr Scenario at Low speed Correlation is suprerior
Image Processing
False detections are evident

21. Image Processing Is superior
SNR =1 at High Speed
Image Processing Is superior
Correlation
Lock lost at Velocity>0.7square_width
Image Processing

22. Block Diagram Main Function MainControl SearchParams
mainusemaincontrolv4f
Parameter initalization,
Output Video Creation
MainControl
mainusemaincontrolv4f
SearchParams
Determines The next correlation window
Multi Frame Correlator
mfmulticorrelatorv4
Performs 5 window 4 frame correlation
Embeds detection Markers on the Output Video .
PlotResults

23. Algorithm’s Weaknesses
The method of correlating previous frames is limited to deal with low-medium velocities (computational complexity).
In case of higher velocities, IP algorithm is limited in SNR threshold.
Because of the complexity of the correlation method, processing time may become critical.

24. Looking Forward
Improving correlation of previous frames method to reduce calculation time.
Implementation on hardware – achieve real time performance.

25. Bibliography Quantitative Comparison of Algorithms for Tracking Single
Fluorescent Particles
Michael K. Cheezum, William F. Walker, and William H. Guilford
Department of Biomedical Engineering, University of Virginia, Charlottesville, Virginia
22908 USA
2001