Presentation is loading. Please wait.

Presentation is loading. Please wait.

People Movement analysis: trajectories

Similar presentations

Presentation on theme: "People Movement analysis: trajectories"— Presentation transcript:

1 People Movement analysis: trajectories
Behavior analysis is a crucial tool for threat assessment and in general scene understanding Trajectory/path analysis is a first fundamental step for behavior analysis in surveillance: understanding critical and typical paths identify deviations from “normality” collect “occupancy” statistics find suspicious behaviors But also in other multimedia applications Analyze similarities in videos

2 Which are the trajectories that share some specific shape properties?
Problem description Given all the trajectories acquired by a video surveillance system: Which are the trajectories that share some specific location properties? Which are the trajectories that share some specific shape properties? Which are the most frequent Behaviors? Who did perform them? people retrieval

3 Literature on Trajectory analysis
Literature approaches on trajectory comparison can be classified: Depending on the Feature (Point to Point vs Statistical): Adopt a point-to-point comparison or exploit statistical data representation Depending on the Representation (Original vs Transformed): Original feature space or provide a space transformation Depending on the Data Dimension (Complete vs Selected): Use all the temporal data or select a subset

4 Related Works Feature Representation Dimension Point to point
Statistical Original Transformed Complete Selected Distance Basharat08 CVPR08 Gaussian x Hu06 PAMI06 Porikli04 CVPRWs04 HMM HMM cross distance Junejo04 ICPR04 Hausdorf Bashir03 ICIP03 PCA Euclidean Chen08 Sampling Null Space Projection Eigen decomposition PCNSA(Principal Component Null Space analysis) distance Ding08 VLD08 LB_Keogh Shieh08 KDD08 SAX SAX symbol subspace symbol to symbol DTW distance Piotto09 TMM09 Breakpoints Breakpoints quantization symbol to symbol Global Alignment(GA) distance Calderara09 AVSS09 ApproxWrapped LinearGaussian MoAWLG GA KL-divergence pdf distance Picciarelli09 TCMS09 Subsampling SVM Learning

5 References: (Basharat08) Basharat, A. Gritai, and M. Shah. Learning object motion patterns for anomaly detection and improved object detection. In Proc. of IEEE Int’l Conference on Computer Vision and Pattern Recognition, 2008 (Porikli04) F. Porikli and T. Haga. Event detection by eigenvector decomposition using object and frame features. In Proc. Of Computer Vision and Pattern Recognition (CVPR) Workshop,volume 7, pages 114–121, 2004. (Hu06)W. Hu, X. Xiao, Z. Fu, D. Xie, T. Tan, and S. Maybank. A system for learning statistical motion patterns. IEEE Trans. on PAMI, 28(9):1450–1464, September 2006. (Junejo04) Junejo, O. Javed, and M. Shah, “Multi feature path modeling for video surveillance,” in Proc. of Int’l Conference on Pattern Recognition, vol. 2, Aug. 2004, pp. 716– 719. (Bashir03) F. I. Bashir, A. A. Khokhar, and D. Schonfeld, “Segmented trajectory based indexing and retrieval of video data,” in Proc. of IEEE Int’l Conference on Image Processing, 2003, pp. 623–626. (Chen08) X. Chen, D. Schonfeld, and A. Khokhar, “Robust null space representation and sampling for view invariant motion trajectory analysis,” in Proc. of IEEE Int’l Conference on Computer Vision and Pattern Recognition, 2008. (Ding08) H. Ding, G. Trajcevski, P. Scheuermann, X. Wang, and E. J. Keogh, “Querying and mining of time series data: experimental comparison of representations and distance measures,” Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1542–1552, 2008. (Shieh08) Jin Shieh and Eamonn Keogh (2008). iSAX: Indexing and Mining Terabyte Sized Time Series. SIGKDD 2008. (Piotto09) N. Piotto, N. Conci, and F. De Natale. Syntactic matching of trajectories for ambient intelligence applications. IEEE Transactions on Multimedia, 11(7):1266–1275, Nov (Calderara09)S. Calderara, A. Prati, and R. Cucchiara. Learning people trajectories using semi-directional statistics. In Proceedings of IEEE International Conference on Advanced Video and Signal Based Surveillance (IEEE AVSS 2009), Genova, Italy, Sept (Picciarelli08)Piciarelli, C.; Micheloni, C.; Foresti, G.L., "Trajectory-Based Anomalous Event Detection," Circuits and Systems for Video Technology, IEEE Transactions on , vol.18, no.11, pp , Nov. 2008

6 Available datasets of trajectories
Various time series (including trajectories): Character Trajectories Data Set: Pen-Based Recognition of Handwritten Digits Data Set: ETISEO project: Soccer player trajectories: “T. D’Orazio, M.Leo, N. Mosca, P.Spagnolo, P.L.Mazzeo A Semi-Automatic System for Ground Truth Generation of Soccer Video Sequences In the Proceeding of the 6th IEEE International Conference on Advanced Video and Signal Surveillance, Genoa, Italy September ” Our own dataset: More than 1000 trajectories of a video surveillance scenario (available at request)

7 Trajectory analysis from two different perspectives
Trajectories are time series of data Querying datasets of time series is a well studied data mining problem which requires: A similarity measure between two time series A clustering technique to classify trajectories In the database-related research the datasets are very large (VLDB) and typically comprise reproducible phenomena (several repetitions of the same class). Thus, similarity measure can be approximated but need to be fast. Clustering can rely on very high number of samples of the same class (simple 1NN clustering often suffices) Viceversa, in video-surveillance research data availability is limited, very diverse from time to time and full of noise. This lack of reproducibility requires a precise measure, also at the cost of computational time. The few data available per class also require more sophisticated clustering approaches Video surveillance scenarios also exhibit a high dinamicity which calls for adaptive methods for classification

8 Ding-Keogh 08 proposal The method proposed in (Ding-Keogh 08) perform the comparison among time series in the original x-y data space. The comparison is performed directly on the original points sequences using dynamic programming and the Dynamic Time Warping Inexact matching such as DTW are required to account for different lengths in time series and for temporal shifts

9 (Ding08) Point-to-point Complete Original
DTW algorithm

10 (Ding08) Point-to-point Complete Original
Each point is compared using the Euclidean distance. Each dimension, namely x and y sequences are compared separately The final distance is the weighted average of the contributions of single dimensions. The Method is effective when comparing similar sequences hence suitable when a large dataset is available, thus suitable for querying VLDB

11 Gullo09 Francesco Gullo, Giovanni Ponti, Andrea Tagarelli, Sergio Greco, A time series representation model for accurate and fast similarity detection, Pages , Pattern Recognition, vol. 42, 11, Nov. 2009 Proposing a new representation of time series based on DSA (Derivative time series Segment Approximation) as dimensionality reduction method and DTW as similarity measure Clustering based on UPGMA (Unweighted Pair Group Method using arithmetic Averages) and classification on KNN Comparison with several similarity measures (DTW, DDTW, LCSS, EDR, etc.) and with several dimensionality reduction methods (SAX, DWT, FWT, etc.). Comparison on 7 public datasets using F-measure

12 Gaussian Model for spatial analysis
Sequence of 2D spatial coordinates Advantages of using spatial coordinates: Embodies additional information about velocity and acceleration Some paths are more common then other depending on their position on the scene Represent partially the reaction of people to the structure of the scenario

13 Gaussian Model for spatial analysis
Due to the uncertainties on the measure of points coordinates we choose a Gaussian model to model every point location Bivariate Gaussian Centered on point coordinate having fixed variance.

14 Mapping Gaussians to Symbols
A single trajectory is modeled as a sequence of point Coordinates: On each point a Spatial Gaussian pdf is fitted. Trajectory model is then represented as a sequence of symbols . Where

15 Clustering Trajectories
Positional Gaussian Clustering Frequent and anomalous behaviors can be obtained by clustering trajectories: According to positions and detect the most frequent activity zones (Gaussian model)

16 On-line Trajectories Classification
Additionally trajectories can be classified on-line and anomalous paths detected. Normal Clusters Abnormal

17 Morris-Trivedi survey on trajectory analysis
B. Morris and M. Trivedi, “A survey of vision-based trajectory learning and analysis for surveillance,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1114–1127, Aug

18 Morris-Trivedi survey on trajectory analysis
B. Morris and M. Trivedi, “A survey of vision-based trajectory learning and analysis for surveillance,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1114–1127, Aug

19 Morris-Trivedi survey on trajectory analysis

20 Trajectory shape analysis
Trajectory shape analysis for “abnormal behavior” recognition in video surveillance. Different context than VLDB: few and noisy data, high degree of variability, tracking errors Trajectory Shape similarity; invariant to space shifts Not only space-based or time-based similarity

21 Trajectory Shape Analysis by angles
Sequence of 2D spatial coordinates Sequence of 1D angles Advantages of using angles: more compact representation invariant to spatial translations (both local and global), thus describing trajectory shape

22 Imagelab Proposal Trajectory description with angle sequence
Statistical representation with a Mixture of Von Mises Distributions (MovM) Coding with a sequence of selected vM pdf identifiers Code Alignment Clustering with k-medoids A. Prati, S. Calderara, R. Cucchiara, "Using Circular Statistics for Trajectory Analysis" in Proceedings of CVPR 2008 Definition of EM algorithm for MovM Using Dynamic programming Definition of Bhattacharyya distance fon vM and on-line EM

23 Training set and on-line classification
Clustering with Br distance Alignement Trajectory clusters repository MovM(Tj) EM for MoVM Trajectory repository Coding with MAP <S={S1j..Snjj},MovM(Tj)> On-line EM for MoVM Classificationwith Br distance Surveillance system Normal/ abnormal

24 Von Mises distribution
When the variables represent angles, Gaussians or MoGs are inappropriate. Example: two observations at 1° and 359°. Modeling these data with a univariate Gaussian distribution is incorrect. In fact, if we select the origin at 0°  if we select the origin at 180°  Von Mises distribution is more suitable to treat periodic variables, being circularly defined I0 = modified zero-order Bessel function of the first kind

25 Mixture of von Mises and Mixture of Gaussians (MoG)
MovM: MoG:

26 Modelling a single trajectory
1)A single trajectory is modeled as a sequence of angles: 2) A specifically defined EM algorithm is used:

27 EM for MovM distribution
Likelihood of complete data set: Expected value of the log likelihood: E-step: estimate of the responsabilities:

28 EM for MovM distribution
M-step: maximizing wrt : M-step: maximizing wrt : function zeros found by inverted numerically

29 Mapping angles to symbols
2) A single trajectory is modeled as a sequence of angles and after having defined the MoVM as a sequence of symbols:

30 Distance for sequences
We transform a comparison between two sequences of either angles or coordinates in the comparison between two sequences of symbols, with each symbol corresponding to the proper probability distribution However, due to acquisition noise, uncertainty and spatial/temporal shifts, exact matching between sequences is unsuitable for computing similarity We use global alignment between two sequences, basing the distance as a cost of the best alignment of the symbols Dynamic programming techniques are used to speed up the process.

31 Global vs local alignment
Global alignment Global vs local alignment Using global alignment instead of local one is preferable because the former preserves both global and local shape characteristics Dynamic programming is used to reduce computational time to O (ni · nj), where ni and nj are the lengths of the two sequences.

32 Inexact matching Since the symbols we are comparing correspond to pdf, match/mismatch should be proportional to the distance between the two corresponding pdfs Need to evaluate distance between two pdfs: Angular: Von Mises Distributions Bhattacharyya distance bw pdfs (closed form) Spatial: Gaussians Distributions Bhattacharyya distance bw pdfs ( )

33 where cB is the Bhattacharyya coefficient
Sequence similarity where cB is the Bhattacharyya coefficient The best alignment is then converted in a distance and used for clustering and testing

34 Comparison of alignment techniques
When the sequences are characterized by different lengths, DTW tries to stretch the two sequences in order to find the optimal time warping path with the consequence of eventually adding additional matches. Global alignment (based on Needleman-Wunsch algorithm), on the other hand, simply adds gaps to align the sequences leading to the advantage of being more susceptible to slight time series’ changes by controlling the gap cost value

35 Comparison of alignment techniques

36 Clustering trajectories
The distance is used to cluster the trajectories in the training set either according their shape or they location k-medoids algorithm: prototype of the cluster is the element that minimizes the sum of intra-class distances To compute the best number of k clusters, iterative k-medoids: initialization: i = 0, k(0) = Nt (cardinality training set); each trajectory is chosen as medoid) of the cluster Step 1: Run k-medoids algorithm with k(i) clusters Step 2: If there are two medoids with a similarity greater than a threshold Th, merge them and set k(i+1) = k(i)−1. Increment i and go back to step 1.

37 Experimental Result We report results on a corpus of 3000 trajectories with an average length of 100 points We compare our method with the baseline off-line time sequence comparison method of [Keog02] E. Keogh., “Exact indexing of dynamic time warping,” in 28th International Conference on Very Large Data Bases. Hong Kong, 2002, pp. 406–417 Method Classification Accuracy Normal Abnormal Accuracy Online VM + GA 96% 97% Gaussian + Online GA 93% [Keog02] on complete trajectory 85% 87%

38 Comparison between VS and VLDB approaches

39 Comparison between VS and VLDB approaches
Results on synthetic dataset

40 Comparison between VS and VLDB approaches
Results on real dataset

41 Adding the speed Pure trajectory shape is not sufficiently always discriminative in surveillance scenarios: the same path covered by a walk or by a run has a different meaning in terms of behavior Add the speed to the shape description to provide a more complete analysis of the trajectory.

42 Trajectory encoding For each couple of subsequent point the angle θ and the velocity vector module ρ are computed For each couple of parameters (θi, ρi) the encoding is performed using a polar scheme Velocity module is used to choose the ring and the direction is used to choose the sector

43 Alignment score for trajectory comparison
After the polar encoding a trajectory Ti is then represented as a sequence of literals S={si,1,si,2,si,3…} We define a suitable score to compare people trajectories given two simbols sp,i and sq,j and the corresponding codes ca1,b1 and ca2,b2 The matching score λi,j is finally normalized to 1 and the similarity metric ξi,j is computed

44 Experiments We log for training 88 trajectories from the multicamera system at our campus during ordinary working days We collect 121 trajectories for testing purposes being labeled manually by an expert as belonging to one of the 12 clusters previously computed The classification rate is 74%. Most of errors are due to two main factors: First: lack of data in the training set Second: inherent difficulties for the expert to answer the question “Which is the most similar trajectory in the direction and the velocity domain? ”

45 Experiments Error example:
S. Calderara, R. Cucchiara, A. Prati, "A Dynamic Programming Technique for Classifying Trajectories" in Proceedings of IEEE International Conference on Image Analysis and Processing (IEEE ICIAP 2007), Modena, Italy, pp.  , Sept , 2007

46 Estimation of precision m in Von Mises pdf is troublesome
Trajectory modeling Use of semi-directional statistics to jointly model linear (speed) and circular (direction) data Estimation of precision m in Von Mises pdf is troublesome Using a approximated wrapped Gaussian pdf is preferable: Similar treatment of its linear counterpart a linear approximation of the variance parameter even for circular variables: Gaussian MLE to compute the joint multivariate covariance matrix

47 Checking independence
since directions and speed are dependent:

48 State of the Art approaches
H. Ding, G. Trajcevski, P. Scheuermann, X. Wang, and E. J. Keogh, “Querying and mining of time series data: experimental comparison of representations and distance measures,” Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1542–1552, N. Piotto, N. Conci, and F. De Natale. Syntactic matching of trajectories for ambient intelligence applications. IEEE Transactions on Multimedia, 11(7):1266–1275, Nov We choose to test our MoAWLG method against two state of the art approaches: Point-to-point, Complete, Original: (Ding-Keogh08) (same as before, but with also speed) Point-to-point, Selected, Transformed: (Piotto09)

49 (Piotto09) Point-to-point Selected Quantized
The method proposed in (Piotto 09) perform the comparison among selected quantize representations of the original position-speed dataspace. Characteristic points of the sequences (breakpoints) are extracted: Temporal Breakpoints: consecutive points in a small area are represented by a single point associated with the time interval the objects stays in its position Spatial Breakpoints: sudden(a) or slow curvature changes(b) are selected as representative points of the trajectory.

50 (Piotto09) Point-to-point Selected Quantized (2)
Once the breakpoints B are computed two consecutive breakpoints identifies a segment. Every segment is then associated to a symbol Where :

51 (Piotto09) Point-to-point Selected Quantized (3)
Every Symbols’ values are quantized and associated to literals: Directions are quantized not uniformly Speed and time are quantized in fixed intervals Symbols’ sequences are aligned using Global Alignment separately for every dimension (direction,speed,time) and the final similarity score is a weighted sum of partial scores.

52 Experimental comparison
We compare our AWLG method with the approaches in (Ding08) and (Piotto09) on a dataset of about 500 trajectories manually ground truthed and divided in clusters We perform 4 tests: T1 and T2: ordinary days acquired trajectories T3: Actor played straight trajectories T4: T3 Trajectories at different speeds.

53 Experimental comparison
Clustering accuracy was measured using the same K-medoids based clustering on distance matrices computed with the different methods described Test ID Number of Trajectories (Ding08) (Piotto09) Our Approach T1 140 78% 73% 95% T2 108 80% 87% 99% T3 145 94% 86% 96% T4 100 90% 97%

54 Conclusions Trajectory analysis is one of the most powerful task to compare movements of people many and many different proposals for large datasets of long trajectories typical data series comparisons point to point and complete could be preferable With smaller and noisy dataset statistical methods could be the best ones With MoG for spatial representation With MoVM for shape representation only With MoAWLG for shape and speed representation

55 Multiple camera and distributed tracking
Multi-camera tracking with camera with overlapping FOVs: Use calibration and 3D geometry Improve with Probabilistic Association Distributed Tracking with camera without overlapping FOVs: Search for similarity Content based retrieval methods Global descriptors: Histograms texture.Medioni’s circular histograms, Mixture of gaussians… .. S. Calderara, R.Cucchiara, A. Prati Multimedia Surveillance: Content based Retrieval with Multicamera People Tracking Proc of VSSN 2006

56 For any other information
Rita Cucchiara Dipartimento di Ingegenria dell’Informazione Thanks to Imagelab Andrea Prati, Roberto Vezzani, Costantino Grana, Simone Calderara, Giovanni Gualdi, Paolo Piccinini, Paolo Santinelli, Daniele Borghesani, Davide Baltieri, Sara Chiossi, Rudy Melli, Emanuele Perini, Giuliano Pistoni..

Download ppt "People Movement analysis: trajectories"

Similar presentations

Ads by Google