1. Yamini Balasukumaran Rekha Dilli Babu
KLT(Kanade-Lucas-Tomasi) feature tracking algorithm in embedded hardware
Yamini Balasukumaran
Rekha Dilli Babu

2. Project Goal Track mice using embedded hardware
Implement KLT algorithm for tracking
Efficient embedded system implementation

3. Project Plan Study the theory for KLT (Good features to track).
Study the given KLT implementation details(Vision group at Stanford University)
Familiarize with the TMS320C5510 DSP kit.
Implement the KLT in TMS320 to track mice.

4. KLT Algorithm Theory Good features to track
Feature selection deals with extracting attributes that result in some quantitative information of interest.
Tracker does not track single pixels but windows of pixels.
Good window is one that can be tracked well.
The goodness of each pixel is measured as the minimum eigenvalue of the 2 by 2 gradient matrix computed from the tc->window_width by tc->window_height window around the pixel
Eigen values indicate whether there is a constant intensity profile, unidirectional pattern or salt and pepper textures.

5. KLT Algorithm Implementation
Functions: Has functions to select good features, track the selected features from one frame to another(KLTSelectGoodFeatures,KLTTrackFeatures).
Can customize the tracker by manually setting various parameters like mindist, Window_width, window_height, min_eigenvalue, subsampling.
Structures: KLT_TrackingContext (to customize tracker), KLT_Feature, KLT_FeatureList.

6. KLT Implementation Details
KLT_Feature
typedef struct {
KLT_locType x;
KLT_locType y;
int val;
} *KLT_Feature;
KLT_TrackingContext
typedef struct {
int mindist;
int window_width, window_height;
KLT_BOOL sequentialMode;
KLT_BOOL smoothBeforeSelecting;
KLT_BOOL writeInternalImages;
int min_eigenvalue;
float min_determinant;
float min_displacement;
int max_iterations;
float max_residue; :
void *pyramid_last_grady;
} *KLT_TrackingContext;

7. KLT Implementation Details KLTSelectGoodFeatures
Gradients are computed in x and y direction and are used to select the features
tc-skippedPixels can be used to speed up the process
Goodness of pixel measured as minimum Eigen value of gradient matrix
Pixels are sorted in ascending order of goodness
nFeatures are selected
Meet mindist and min_eigen value.
KLTSelectGoodFeatures()
void KLTSelectGoodFeatures(
KLT_TrackingContext tc,
KLT_PixelType *img,
int ncols,
int nrows,
KLT_FeatureList fl);

8. Hardware - TMS320C5510 DSK kit
Chosen because of availability with the idea that the design could be ported to another appropriate hardware.
TMS320C5510 DSK kit has fixed point DSP processor(200 Mhz ), has support for DMA.
Supports Multiply accumulate hardware(MAC) instruction - intrinsic functions sum += (*ppp++) * (kernel.data[k]); sum = _smac(sum,*ppp++,kernel.data[k]) ;
On chip hardware timer .

9. Implementation of Feature selection (Embedded system)
Measured the execution time of the KLTSelectFeatures function using the high resolution timer functions.
Start = CLK_gethtime();
functioncall();
stop = CLK_gethtime();
time_used = stop - start;
num_clocks_count = CLOCK_FREQ_MS/CLK_countspms();
num_clocks_used = num_clocks_count * time_used ;
tottime = (num_clocks_used/200000);
In the case of 45 by 22 frame image this value was 51ms for the KLTSelectFeatures function.

10. Implementation of Feature selection (Embedded system)
File-IO execution time (3 usec) was found to be negligible compared to the Select-features function(51 ms).
The periodic function scheduled for 70 ms. (14 frames per sec)
Separate the signal processing functions from the file formatting functions. If implemented on a system like C5471 which has an ARM and DSP then it would be easier to implement.
Periodic-function
Generator
(every 70 ms)
Image-file Pre-processor
KLt-Feature-Selection

11. Results Location X1 Y1 X2 Y2 KLT reference implementation 22 13 31 13
Embedded implementation

12. Tracking mice

13. Scope for improvement Choose a Floating point processor.
Set-up which has streaming data interface
Parallel processing to improve speed up (e.g.. for computing horizontal and vertical gradients)
Change the algorithm to save on memory allocations.

14. Conclusion
Implemented the feature selection portion of the algorithm with limited scope.

15. References http://vision.stanford.edu/~birch/klt/
Jianbo Shi and Carlo Tomasi. Good Features to Track. IEEE Conference on Computer Vision and Pattern Recognition, pages , 1994.
TMS320C5510 Reference Manuals