Belief Networks in Computer Vision Applications Alex Yakushev CMPS 290C final project Winter 2006

Kalman Filters Predicted position = (Previous Position)* (Motion model ) + Noise True position = observed position + Noise Extrapolate object position by taking a weighted average of the two. Only depends on current observation and previous state

Kalman Filter as a Belief Network xkxk x k+1 zkzk z k+1 Motion model Prediction covariance Model Noise State model Observation noise

Kalman Filters Becomes a simple update rule

Something completely different ● Are old fashioned Bayesian networks of any use? ● What if the object we are trying to track does not have a constant velocity or acceleration? ● Sherrah and Gong have designed a BN to track hand and face movements.

Something completely different (Sherrah, Gong)

Results (Sherrah, Gong) ● For their setup they had 70% of frames classified correctly without any contextual knowledge ● This improved to 78% with a Kalman filter ● 87% with their method......but Assumptions are unreasonable: 1. the subject is oriented roughly towards the camera for most of the time 2. the subject is wearing long sleeves 3. reasonably good color segmentation of the head and hands is possible, and 4. the head and hands are the larges moving skin color clusters in the image

“Dynamic Bayesian Network” (Pavlovic, Rehg, Cham, Murphy) State variables s t belong to a set of S discrete symbols.

Dynamic Bayesian Network ● Two hidden variables (x t, s t ) ● If x 0 is Gaussian then x 1 is a mixture of S Gaussians, x 2 is a mixture of S 2 Gaussians... ● No “simple” update rule ● Exact inference is expensive (Pavlovic, Rehg, Cham, Murphy)

Conclusions ● As usual, increase in expressive power costs computation time ● People create custom networks, and either give a solution for the specific structure, or use approximate inference ● No silver bullet, good results possible with both simple and complex network structures ● Using a simpler structure (Markov Network) may work for many applications

References ● Jamie Sherrah, Shaogang Gong, Tracking Discontinuous Motion Using Bayesian Inference, Lecture Notes in Computer Science, Volume 1843, Jan 2000, Pages 150 – 166 ● V. Pavlović, J. M. Rehg, T. J. Cham, and K. P. Murphy, A dynamic Bayesian network approach to figure tracking using learned dynamic models, in International Conference on Computer Vision, Corfu, Greece, September ● Murphy, K., Dynamic Bayesian Networks: Representation, Inference and Learning. PhD thesis, UC Berkeley, Computer Science Division (2002) ● Hai Tao, Object Tracking and Kalman Filtering, CMPE264 Lecture notes,