Oklahoma State University Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis Xin Fan and Guoliang Fan Visual Computing and Image Processing Lab School of Electrical and Computer Engineering Oklahoma State University 4th Joint IEEE International Workshop on Object Tracking and Classification Beyond the Visible Spectrum(OTCBVS'07) Minneapolis, MN, USA, June 22, 2007

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 2 Problem Statement Introduction Problem Statement Related Work Generative model Experimental Results Conclusions  Motion models  Deal with object movements  Why important?  complex motion patterns, e.g., maneuvering  no good appearance model, e.g., low SNR  provide good prediction for robust and efficient tracking  Challenges  Hardly predict maneuvering actions  Model constraints  A motion model that incorporates constraints

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 3  Our observations  Maneuvering actions are due to forces and torques.  forces and torques cause kinematic changes  Newton equations for rigid body motion  Rigid body motion VS point motion  Newton equations  Forces are dependent on kinematics  Limited output power of engines.  Uncertainties exist, e.g., air resistance, road friction, mechanical instability, etc. Introduction Problem Statement Related Work Generative model Experimental Results Conclusions

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 4 Problem Formulation Introduction Problem Statement Problem Formulation Related Work Generative model Experimental Results Conclusions  Switching statistical models for maneuvering variables (forces and torques)  Maneuvering actions are due to forces and torques.  Newton equations to define kinematics evolution densities  Newton equations of rigid body motion  Rayleigh distribution to model velocity-force constraints  Physical constraints reveal how forces are dependent on kinematics.  Organize these dependencies with a probabilistic graphical model

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 5 Related work  White Gaussian noise acceleration (WGNA)  Point target assumption  Miller’s condition mean estimation  Rigid Newton dynamics  Jump-diffusion process, not sequentially  Switching Linear dynamic system (SLDS) or Jump Markov linear system (JMLS)  Discrete switching variables for maneuvering actions  No explicit physical dynamics  Inference algorithms  IMM works for Gaussian densities  BP works for tree structures  Sampling based approximation Introduction Problem Statement Problem Formulation Related Work Generative model Experimental Results Conclusions

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 6 Generative Model - Structure Introduction Problem Statement Problem Formulation Related Work Generative model Structure Experimental Results Conclusions  Generative model  How forces and torques generate kinematics changes  how kinematics generate observations Frames Velocity Position Orientation Forces Torques

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 7 Generative model-Cause variables Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Experimental Results Conclusions  Switching continuous probabilistic models  Specify three switching normal distributions for forces.  Ternary uniform mixture for torques (angular velocity)

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 8 Generative model – Temporal constraints Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Experimental Results Conclusions  Newton equations  Investigate the dynamics of 3D rigid motion  Define the kinematic dependence by Newton equations of rigid body motion.

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS'07. 9 Generative model – Temporal constraints Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Experimental Results Conclusions  Newton equations for 3D rigid motion  Simplified for ground vehicles  p-linear momentum and f- force  h - angular momentum and τ- torque

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Generative model – Temporal constraints Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Experimental Results Conclusions  kinematics dependency via Newton equations Velocity Orientation Position

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Generative model- VF constraints Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Experimental Results Conclusions  Rayleigh distribution for velocity-force constraints  Driving force conditional on velocities  Resistance force conditional on velocities

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Generative model - Likelihood  Simple template matching to define likelihood Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Experimental Results Conclusions

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Generative model - Inference Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference Experimental Results Conclusions  Predict with temporal densities  Evaluate weights with likelihood  MCMC to generate samples of forces  SMC based inference algorithm

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Experiments – Simulated data  Compared with a particle filter (PF) for JMLS  Tracking with coupled linear and angular motion  No coupling Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference Experimental Results Simulated data Conclusions Ours JMPF

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Experiments – Simulated data  Compared with a particle filter (PF) for JMLS  Tracking with coupled linear and angular motion  Has coupling Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference Experimental Results Simulated data Conclusions JMPF Ours

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Experiments – Simulated data  Compared with a particle filter (PF) for JMLS  Tracking with velocity-force constraints Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference Experimental Results Simulated data Conclusions

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Experiments – Real world video  Compared with constant velocity constant turn (CVCT) model Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference Experimental Results Simulated data Real world video Conclusions Ours CVCT

X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis, OTCBVS' Conclusions and future work Introduction Problem Statement Problem Formulation Related Work Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference Experimental Results Simulated data Real world video Conclusions  Conclusions  Graphical model for maneuvering targets, which encode the Newtonian dynamics in a probabilistic framework.  Explicitly and directly build the cause-effect relationship  Feedback constraint from velocity to the forces  Future work  Handle multiple views.  Multiple targets with data association