1. Passive Object Tracking from Stereo Vision Michael H. Rosenthal May 1, 2000

2. Outline Purpose Camera Calibration Data Collection Object Tracking Depth from Stereo Results Conclusions

3. Purpose 3D visualization - blood vessels, organs, teapots Stereo displays require head tracking Cumbersome trackers are undesirable

4. Purpose Passive tracking offers tetherless option Speed and accuracy is a concern

5. How To Do It Mount reflectors on tracking target Calibrate pair of cameras Identify reflectors in each image Calculate positions using stereo disparity Update tracking model using new positions

6. Camera Calibration Select six points on known calibration target Solve for six extrinsic and four intrinsic parameters (ignore distortion) Used derivative of Tsai’s calibration code (www.cs.cmu.edu/~rgw/TsaiDesc.html) Thanks to Herman Towles and Ruigang Yang of Office of the Future for adapted code

7. Data Collection Mounted target on optical rail Translated target at 1mm per frame over 20cm Rotated target through 120 degrees

8. Object Tracking We want to follow an object through a scene Challenge - what targets are best for tracking? Spheres yield spatially ambiguous results Complex shapes limit ambiguity but are hard to track Model of glasses with tracking targets

9. Object Tracking I chose a square and a rectangle - unambiguous, but surprisingly hard to track Square is easy - calculate centroid as weighted average of position and intensity Rectangle is the problem - need center and orientation, makes problem non-trivial Model of glasses with tracking targets

10. Object Tracking Calculate centroid of rectangle Search for shortest axis at some angle through centroid Find edges along short and long axes using derivatives Use endpoints of long axes as tracking targets Use prior results for future frames Model of glasses with tracking targets

11. Depth from Stereo Use point pairings to get depth Find shortest segment between the two pixel rays Use midpoint as position estimate Trucco section 7.4 describes equations

12. Results Translation Moved stage by 210mm Average measured translation: 178mm 16% error Rotation Rotated target by 60 degrees Averaged measured rotation: 70 degrees 16% error Neither showed high noise, so systematic error is likely (calibration?)

13. Conclusions Camera calibration is somewhat challenging Good camera calibration is very challenging Robust object tracking will require significant development Simplified targets will reduce complexity