1. Pursuit Evasion Games and Multiple View Geometry
René Vidal
UC Berkeley
Part I: Motivation, 10 minutes, 9 slides -> 8
Part II: Previous work, 35 minutes, 21 slides -> 20
Part II: Future work, 15 minutes, 8 slides -> 10

2. “Research Trends” Control Computer Vision
Hybrid Systems, Embedded Systems
Quantum Control, Multi-Agent Control
Computer Vision
Multi-body Structure from Motion
Omnidirectional Vision
Many more …
Texture, segmentation and grouping, shape, stereo, illumination, recognition, real-time vision

3. Previous Work Robotics and Control Computer Vision
Pursuit-Evasion Games (ICRA’01-CDC’01-TRA’01)
Decidable Classes of Discrete Time Hybrid Systems (HSCC’00-CDC’01)
Computer Vision
Multiple View Geometry (IJCV’01-ECCV’02)
Optimal Motion Estimation (ICCV’01)
Planar motions with Small Baselines (ECCV’02)
Camera Self-Calibration (ECCV’00-IJCV’01)

4. Pursuit-Evasion Games
Cory Sharp, Omid Shakernia, David Shim, Jin Kim
Shankar Sastry

5. Pursuit-Evasion Game Scenario
Evade!

6. PEG: Map Building (Hespanha CDC’99)
Given measurements build a probabilistic map
Measurement step: model for sensor detection
Prediction step: model for evader motion

7. PEG: Pursuit Policies (Hespanha CDC’99)
Greedy Policy
Pursuer moves to the reachable cell with the highest probability of having an evader
The probability of the capture time being finite is equal to one (Hespanha CDC ’99)
The expected value of the capture time is finite (Hespanha CDC ’99)
Global-Max Policy
Pursuer moves to the reachable cell which is closest to the cell in the whole map with the highest probability of having an evader
May not take advantage of multiple pursuers (all the pursuers could move to the same place)

8. PEG: Control Architecture (ICRA’01)
position of evader(s)
position of obstacles
strategy planner
position of pursuers
map builder
communications network
evaders
detected
obstacles
pursuers
positions
Desired pursuers
positions
tactical
planner
trajectory
regulation
tactical planner & regulation
actuator
positions
[4n]
lin. accel.
& ang. vel.
[6n]
inertial
[3n]
height over terrain [n]
obstacles detected
evaders detected
vehicle-level sensor fusion
state of helicopter & height over terrain
obstacles detected
control signals
[4n]
agent dynamics
actuator
encoders
INS
GPS
ultrasonic altimeter
vision
Exogenous
disturbances
terrain
evader

9. PEG: Vision System

10. PEG: Experimental Results (Summer’00)

11. PEG: Architecture Implementation (CDC’01)
Navigation Computer
Serial
Vision Computer
Strategic Planner
Helicopter Control
GPS: Position
INS: Orientation
Camera Control
Color Tracking
UGV Position Estimation
Communication
UAV Pursuer
Map Building
Pursuit Policies
Communication
Runs in Simulink
Same for Simulation and Experiments
TCP/IP
Serial
Robot Micro Controller
Robot Computer
Robot Control
DeadReck: Position
Compass: Heading
Camera Control
Color Tracking
GPS: Position
Communication
UGV Pursuer
UGV Evader

12. PEG: Experimental Results (Spring’01)

13. PEG: Experimental Results (Spring’01)
Add simulation video before

14. PEG: Experimental Results (Spring’01)
Pursuit Policy v.s. Vision System

15. Vision Based Landing

16. Landing: (Sharp & Shakernia ICRA’01)

17. PEG: Summary
Proposed a probabilistic framework and a hierarchical control architecture for pursuit-evasion games
Developed algorithms for
Sensor fusion, helicopter control, vision based detection, vision based landing
Implemented architecture with UGVs and UAVs in real-time

18. Multiple View Geometry
The Multiple View Matrix
Yi Ma (UIUC), Jana Kosecka (GMU), Shankar Sastry (UCB)

19. Multiple View Geometry (MVG)

20. Multiple View Geometry (MVG)
Obtain camera motion and scene structure from multiple images of a cloud of 3D feature points

21. MVG: Literature Review
Theory
Two views: Longuet-Higgins’81, Huang & Faugeras’89, …
Three views: Spetsakis et.al.’90, Shashua’94, Hartley’94, …
Four views: Triggs’95, Shashua’00, …
Multi views: Faugeras et.al.’95, Heyden et.al.’97,…
Algorithms
Euclidean: Maybank’93, Weng, Ahuja & Huang’93, …
Affine: Quan & Kanade’96, …
Projective: Triggs’96, …
Orthographic:Tomasi & Kanade’92, …

22. MVG: Anatomy of cases (state of the art)
surface
curve
line
point
theory
algorithm
practice
Euclidean
affine
projective
2 views
3 views
4 views
m views
algebra
geometry
optimization

23. MVG: A need for unification
Euclidean
surface
curve
line
point
2 views
3 views
4 views
m views
theory
algorithm
practice
affine
projective
algebra
geometry
optimization
rank deficiency of
Multiple View Matrix

24. MVG: Formulation Homogeneous coordinates of a 3-D point
Homogeneous coordinates of its 2-D image
Projection of a 3-D point to an image plane
Either Euclidean or Projective

25. MVG: Classical Approach
Given corresponding images of points recover motion, structure and calibration from
This set of equations is equivalent to

26. MVG: The Multiple View Matrix
WLOG choose frame 1 as reference
Theorem: [Rank deficiency of Multiple View Matrix]
(generic)
(degenerate)

27. MVG: Bilinear and Trilinear Constraints
Multiple View Matrix implies bilinear constraints
Multiple View Matrix implies trilinear constraints
Constraints among more than three views are algebraically dependent (quadrilinear in particular)

28. MVG: Degenerate Cases
Theorem [Uniqueness of the pre-image]: Given vectors with respect to camera frames, they correspond to a unique point in the 3-D space if rank(M) = 1. If rank(M) = 0, the point is determined up to a line on which all the camera centers must lie.

29. MGV: Line Features
Point Features
Line Features

30. MVG: Mixed Point and Line Features. . .

32. MGV: Planar Features Point Features Line Features
Besides multilinear constraints, it simultaneously gives homography:

33. Multiple View Landing (ICRA’02)

34. MGV: Multiple View Planar Algorithm Results

35. Optimal Motion Estimation (ICCV’01)
Noisy measurements:
Minimize error:
Subject to:
Lagrange optimization + algebraic elimination:

36. Optimal Motion Estimation (ICCV’01)
M is not a regular Euclidean space
Can do Euclidean optimization, but need to project onto M
There are optimization techniques for Stiefel manifolds
Quadratic convergence is guaranteed if Hessian is non-degenerate [Smith and Brockett ’93]

37. Multiple View Matrix: Summary
Points
M implies bilinear and trilinear constraints
Quadrilinear constraints do not exist
Lines
All indep. constraints are among 3 views
Some are trilinear, some are nonlinear
Mixed points and lines
There are multilinear constraints among 2-3 views
There are nonlinear constraints among 3-4 views

38. Current Work: SFM for curves
Differentiating of a point on a curve
intensity level sets
region boundaries
. . .

39. Current Work: SFM for curves
Multiple view matrix for curves
Gives rise to rank condition
Gives rise to set of ODE’s

40. Current Work: Multi-Body SFM
Geometry of multiple moving objects
Given a set of corresponding image points, obtain:
Number of evaders
Evader to which each point belongs to (segmentation)
Depth of each point
Motion of each pursuer and evader
Orthographic case (Costeira-Kanade’95)
Multiple moving points (Shashua-Levin’01)
Can the multiple view matrix approach be used for motion estimation and segmentation?

41. Current Work: Multi-Body SFM
Generalized Multiple View Matrix [Ma et.al. ’01]
Multiple moving points
Projection

42. Current Work: Multi-Body SFM
Linked Multiple Body Motion Estimation
Infinitesimal motion case
Ursella, Soatto and Perona ‘95
Bregler and Malik ‘98
Discrete motion case
Generalized multiple view matrix
Paden-Kahan inverse kinematic subproblems

43. Current Work: Multi-Body SFM [Vidal’01]
Costeira-Kanade in perspective
Form optical flow matrices
Obtain number of objects
Segment the points

44. Conclusions Computer Vision for Real-Time Control
Pursuit Evasion Games
Vision Based Landing
A new approach to Multiple View Geometry
Simple: just linear algebra
Unifying: Euclidean, projective, 2 views, 3 views, multiple views, points, lines, curves, surface?
New Constraints: There are nonlinear constraints