Presentation is loading. Please wait.

Presentation is loading. Please wait.

Student: Hsu-Yung Cheng Advisor: Jenq-Neng Hwang, Professor

Similar presentations

Presentation on theme: "Student: Hsu-Yung Cheng Advisor: Jenq-Neng Hwang, Professor"— Presentation transcript:

1 Object Tracking, Trajectory Analysis and Event Detection in Intelligent Video Systems
Student: Hsu-Yung Cheng Advisor: Jenq-Neng Hwang, Professor Department of Electrical Engineering University of Washington

2 Outlines Motivation Object Tracking Trajectory Analysis
Event Detection Conclusions and Future Work

3 Motivation Advantage of Video-based systems Applications
Being able to capture a large variety of information Relatively inexpensive Easier to install, operate, and maintain Applications Security surveillance Home care surveillance Intelligent transportation systems There is an urgent need for intelligent video systems to replace human operators to monitor the areas under surveillance.

4 System Modules for Intelligent Event Detection Systems

5 Challenges for Robust Tracking
Segmentation errors Change of lighting conditions Shadows Occlusion Inter-Object Partial/Complete Occlusion, Initial Occlusion, Background Occlusion

6 Inter-Object Occlusion

7 Initial Occlusion

8 Background Occlusion

9 Proposed Tracking Mechanism

10 Background Estimation and Updating
Based on Gaussian mixture models [Stauffer 1999] Model the recent history of each pixel by a mixture of K Gaussian distributions. Every pixel value is checked among the existing K Gaussian distributions for a match. Update the weights for the K distributions and the parameters of the matched distribution The kth Gaussian is ranked by ( ) The top-ranked Gaussians are selected as the background models. Pixel values that belong to background models are accumulated and averaged as the background image. The background image is updated for every certain interval of time.

11 Moving Object Segmentation
Based on background subtraction Fourth order moment [S. Colonnese et al. Proc. of SPIE 2003] Thresholding

12 Kalman Filter Kalman filters are modeled on a Markov chain built on linear operators perturbed by Gaussian noises. At time k, each target has state , where and observation (measurement) Kalman filters are based on linear dynamical systems discretised in the time domain. They are modelled on a Markov chain built on linear operators perturbed by Gaussian noise. The state of the system is represented as a vector of real numbers. At each discrete time increment, a linear operator is applied to the state to generate the new state, with some noise mixed in, ((and optionally some information from the controls on the system if they are known. )) Then, another linear operator mixed with more noise generates the visible outputs from the hidden state. The Kalman filter may be regarded as analogous to the hidden Markov model, with the key difference that the hidden state variables are continuous (as opposed to being discrete in the hidden Markov model). Additionally, the hidden Markov model can represent an arbitrary distribution for the next value of the state variables, in contrast to the Gaussian noise model that is used for the Kalman filter. There is a strong duality between the equations of the Kalman Filter and those of the hidden Markov model. A review of this and other models is given in (Roweis and Ghahramani, 1999) , where Kalman, R. E. "A New Approach to Linear Filtering and Prediction Problems,“ Transactions of the ASME - Journal of Basic Engineering Vol. 82: pp , 1960.

13 Kalman Filter Phases Predict Updated State Estimate Predicted State
Observed Measurements Update

14 Updated State Estimate
Kalman Filter Phases Update Phase Predict Phase Updated State Estimate Predicted State Updated Estimate Covariance Kalman Gain Predicted Estimate Covariance Innovation Covariance (Pk|k): the error covariance matrix (a measure of the estimated accuracy of the state estimate). Innovation (Measurement) Residual

15 Constructing Measurement Candidate List

16 Searching for measurement candidate representation points
Search for q1 and q2 in the two n x n windows centered around p1 and p2, respectively. Compute the dissimilarities between the target object and the potential measurement candidates.

17 Data Association To associate measurements with targets when performing updates Nearest Neighbor Data Association For all the measurement in the validation gate of a target, select the nearest measurement. Probabilistic Data Association (PDA) Joint Probabilistic Data Association (JPDA) Nearest: metric can be euclidean distance, mahanobis distance, etc…

18 Probabilistic Data Association
Consider a single target independently of others y2 denotes the event that the j th measurement belongs to that target. x Combined (Weighted) Innovation y1 y3 Probabilistic Data Association Consider a single target independently of others Y . Bar-Shalom and E. Tse, “Tracking in a cluttered environment with probabilistic data association,” Automatica, vol. 11, pp , Sept

19 Modified PDA for Video Object Tracking
To handle video objects (regions), incorporate the following factor when computing Similarity measure: cross correlation function

20 Experimental Videos

21 Vehicle Tracking Results 1

22 Vehicle Tracking Results 2

23 Human Tracking Results

24 Object Tracking Statistics
Video Sequence 1 Sequence 2 Sequence 3 Sequence 4 Ground Truth 71 64 92 130 Object Detected 72 61 93 128 Miss 3 5 False Alarm 1 Correctly Detected 125 Correctly Tracked 70 58 120 Detection Precision 0.986 1.000 0.989 0.977 Detection Recall 0.953 0.962 Tracking Success Rate 0.951 0.960 Occluded Object Tracking Success Rate: 0.855

25 Trajectory Analysis Class 1 Class 3 Class 5 Class 2 Class 4 Class 6

26 Trajectory Smoothing Perform cubic spline interpolation
Sample the trajectory Perform cubic spline interpolation First of all, we sample the trajectory at a sampling rate determined by the trajectory length and the average speed. Our purpose is to smooth the trajectory but maintain the essential characteristics of the trajectory at the same time. Therefore, if the trajectory is long and the object moves slowly, then a longer sampling period is used. On the other hand, if the trajectory is short or the object moves at a higher speed, a shorter sampling period is adopted.

27 Angle Feature Extraction
Relative Angle Absolute Angle

28 Hidden Markov Model N states Si , i=1,…, N Transition probability aij
Initial probability pi Observation symbol probability bj(k) A complete model l=(A,B,P) A={aij} B={bjk} P={pi} Sunny Cloudy Rainy P(walk) = 0.5 P(bike) = 0.4 P(bus) = 0.1 P(walk) = 0.4 P(bike) = 0.3 P(bus) = 0.3 P(walk) = 0.2 P(bike) = 0.1 P(bus) = 0.7

29 Example of HMM Observation sequence O = { walk, bike, bus, bus, bike, walk, … }

30 Three Problems in HMM Given l, compute the probability that O is generated by this model How likely did O happen at this place? Given l, find the most likely sequence of hidden states that could have generated O How did the weather change day-by-day? Given a set of O, learn the most likely l Train the parameters of the HMM forward-backward algorithm Viterbi algorithm Baum-Welch algorithm

31 Left-to-right HMM for Trajectory Classification

32 K-means Clustering of Feature Points

33 Number of Training and Test sequences
Video for both training and testing Trajectory Class Training Trajectories Testing Objects Testing Trajectories Class 1 12 64 307 Class 2 11 18 66 Class 3 13 27 Class 4 5 20 Class 5 8 26 32 Class 6 29 45 Video for testing only When performing classification, the partial trajectories are tested after an object appears in the ROI for more than 90 frames. After that, the trajectory of the object is updated and tested every 5 frames. Therefore, for an object staying in the video scene for more than 90 frames, more than one trajectory will be created as the test trajectories

34 Trajectory Classification Statistics
Accuracy Class 1 307 100% Class 2 64 2 97.4% Class 3 25 92.6% Class 4 20 Class 5 1 31 96.8% Class 6 43 95.5%

35 Anomalous Trajectories

36 Event Detection Type I Events Type II Events Type III Events
Simple rule-based decision logic Entering a dangerous region Stopping in the scene Driving on the road shoulder Type II Events Based on trajectory classification results via HMM using angle features Illegal U-turns or left turns Anomalous trajectories Type III Events Based on trajectory classification results via HMM using speed features Speed change

37 Conclusions and Future Works
Tracking Kalman filtering for prediction Modified PDA for data association Basic Events Simple rule-based decision logic HMM Higher Level Events Combining basic events More flexible models


Download ppt "Student: Hsu-Yung Cheng Advisor: Jenq-Neng Hwang, Professor"

Similar presentations

Ads by Google