Presentation is loading. Please wait.

Presentation is loading. Please wait.

ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy People Movement analysis: trajectories Behavior analysis is a crucial tool.

Similar presentations


Presentation on theme: "ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy People Movement analysis: trajectories Behavior analysis is a crucial tool."— Presentation transcript:

1 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy People Movement analysis: trajectories Behavior analysis is a crucial tool for threat assessment and in general scene understanding Trajectory/path analysis is a first fundamental step for behavior analysis in surveillance: understanding critical and typical paths identify deviations from normality collect occupancy statistics find suspicious behaviors But also in other multimedia applications Analyze similarities in videos

2 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Problem description Given all the trajectories acquired by a video surveillance system: Which are the most frequent Behaviors? Which are the trajectories that share some specific shape properties? Which are the trajectories that share some specific location properties? Who did perform them? people retrieval

3 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Literature on Trajectory analysis Literature approaches on trajectory comparison can be classified: Depending on the Feature (Point to Point vs Statistical ): Adopt a point-to-point comparison or exploit statistical data representation Depending on the Representation (Original vs Transformed): Original feature space or provide a space transformation Depending on the Data Dimension (Complete vs Selected): Use all the temporal data or select a subset

4 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Related Works FeatureRepresentationDimension Point to point StatisticalOriginalTransformedCompleteSelectedDistance Basharat08 CVPR08 GaussianxxStatistical Hu06 PAMI06 GaussianxxStatistical Porikli04 CVPRWs04 HMMxxHMM cross distance Junejo04 ICPR04 xxxHausdorf Bashir03 ICIP03 xPCA Euclidean Chen08 CVPR08 Sampling Null Space Projection Eigen decompositi on PCNSA(Principal Component Null Space analysis) distance Ding08 VLD08 xxxLB_Keogh Shieh08 KDD08 xSAX SAX symbol subspace symbol to symbol DTW distance Piotto09 TMM09 xBreakpoints Breakpoints quantization symbol to symbol Global Alignment(GA) distance Calderara09 AVSS09 ApproxWrapped LinearGaussian MoAWLGx GA KL-divergence pdf distance Picciarelli09 TCMS09 xxSubsamplingSVM Learning

5 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy References: (Basharat08) Basharat, A. Gritai, and M. Shah. Learning object motion patterns for anomaly detection and improved object detection. In Proc. of IEEE Intl Conference on Computer Vision and Pattern Recognition, 2008 (Porikli04) F. Porikli and T. Haga. Event detection by eigenvector decomposition using object and frame features. In Proc. Of Computer Vision and Pattern Recognition (CVPR) Workshop,volume 7, pages 114–121, (Hu06)W. Hu, X. Xiao, Z. Fu, D. Xie, T. Tan, and S. Maybank. A system for learning statistical motion patterns. IEEE Trans. on PAMI, 28(9):1450– 1464, September (Junejo04) Junejo, O. Javed, and M. Shah, Multi feature path modeling for video surveillance, in Proc. of Intl Conference on Pattern Recognition, vol. 2, Aug. 2004, pp. 716– 719. (Bashir03) F. I. Bashir, A. A. Khokhar, and D. Schonfeld, Segmented trajectory based indexing and retrieval of video data, in Proc. of IEEE Intl Conference on Image Processing, 2003, pp. 623–626. (Chen08) X. Chen, D. Schonfeld, and A. Khokhar, Robust null space representation and sampling for view invariant motion trajectory analysis, in Proc. of IEEE Intl Conference on Computer Vision and Pattern Recognition, (Ding08) H. Ding, G. Trajcevski, P. Scheuermann, X. Wang, and E. J. Keogh, Querying and mining of time series data: experimental comparison of representations and distance measures, Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1542–1552, (Shieh08) Jin Shieh and Eamonn Keogh (2008). iSAX: Indexing and Mining Terabyte Sized Time Series. SIGKDD (Piotto09) N. Piotto, N. Conci, and F. De Natale. Syntactic matching of trajectories for ambient intelligence applications. IEEE Transactions on Multimedia, 11(7):1266–1275, Nov (Calderara09)S. Calderara, A. Prati, and R. Cucchiara. Learning people trajectories using semi-directional statistics. In Proceedings of IEEE International Conference on Advanced Video and Signal Based Surveillance (IEEE AVSS 2009), Genova, Italy, Sept (Picciarelli08)Piciarelli, C.; Micheloni, C.; Foresti, G.L., "Trajectory-Based Anomalous Event Detection," Circuits and Systems for Video Technology, IEEE Transactions on, vol.18, no.11, pp , Nov. 2008

6 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Available datasets of trajectories Various time series (including trajectories): Character Trajectories Data Set: Pen-Based Recognition of Handwritten Digits Data Set: ETISEO project: Soccer player trajectories: T. DOrazio, M.Leo, N. Mosca, P.Spagnolo, P.L.Mazzeo A Semi-Automatic System for Ground Truth Generation of Soccer Video Sequences In the Proceeding of the 6th IEEE International Conference on Advanced Video and Signal Surveillance, Genoa, Italy September Our own dataset: More than 1000 trajectories of a video surveillance scenario (available at request)

7 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Trajectory analysis from two different perspectives Trajectories are time series of data Querying datasets of time series is a well studied data mining problem which requires: A similarity measure between two time series A clustering technique to classify trajectories In the database-related research the datasets are very large (VLDB) and typically comprise reproducible phenomena (several repetitions of the same class). Thus, similarity measure can be approximated but need to be fast. Clustering can rely on very high number of samples of the same class (simple 1NN clustering often suffices) Viceversa, in video-surveillance research data availability is limited, very diverse from time to time and full of noise. This lack of reproducibility requires a precise measure, also at the cost of computational time. The few data available per class also require more sophisticated clustering approaches Video surveillance scenarios also exhibit a high dinamicity which calls for adaptive methods for classification

8 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Ding-Keogh 08 proposal The method proposed in (Ding-Keogh 08) perform the comparison among time series in the original x-y data space. The comparison is performed directly on the original points sequences using dynamic programming and the Dynamic Time Warping Inexact matching such as DTW are required to account for different lengths in time series and for temporal shifts

9 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy (Ding08) Point-to-point Complete Original DTW algorithm

10 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy (Ding08) Point-to-point Complete Original Each point is compared using the Euclidean distance. Each dimension, namely x and y sequences are compared separately The final distance is the weighted average of the contributions of single dimensions. The Method is effective when comparing similar sequences hence suitable when a large dataset is available, thus suitable for querying VLDB

11 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Gullo09 Francesco Gullo, Giovanni Ponti, Andrea Tagarelli, Sergio Greco, A time series representation model for accurate and fast similarity detection, Pages , Pattern Recognition, vol. 42, 11, Nov A time series representation model for accurate and fast similarity detection Proposing a new representation of time series based on DSA (Derivative time series Segment Approximation) as dimensionality reduction method and DTW as similarity measure Clustering based on UPGMA (Unweighted Pair Group Method using arithmetic Averages) and classification on KNN Comparison with several similarity measures (DTW, DDTW, LCSS, EDR, etc.) and with several dimensionality reduction methods (SAX, DWT, FWT, etc.). Comparison on 7 public datasets using F-measure

12 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Gaussian Model for spatial analysis Sequence of 2D spatial coordinates Advantages of using spatial coordinates: Embodies additional information about velocity and acceleration Some paths are more common then other depending on their position on the scene Represent partially the reaction of people to the structure of the scenario

13 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Gaussian Model for spatial analysis Due to the uncertainties on the measure of points coordinates we choose a Gaussian model to model every point location Bivariate Gaussian Centered on point coordinate having fixed variance.

14 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Mapping Gaussians to Symbols A single trajectory is modeled as a sequence of point Coordinates: On each point a Spatial Gaussian pdf is fitted. Trajectory model is then represented as a sequence of symbols. Where

15 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Clustering Trajectories Frequent and anomalous behaviors can be obtained by clustering trajectories: According to positions and detect the most frequent activity zones (Gaussian model ) Positional Gaussian Clustering

16 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy On-line Trajectories Classification Additionally trajectories can be classified on-line and anomalous paths detected. Normal Clusters Abnormal

17 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Morris-Trivedi survey on trajectory analysis B. Morris and M. Trivedi, A survey of vision-based trajectory learning and analysis for surveillance, IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1114–1127, Aug

18 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Morris-Trivedi survey on trajectory analysis B. Morris and M. Trivedi, A survey of vision-based trajectory learning and analysis for surveillance, IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1114–1127, Aug

19 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Morris-Trivedi survey on trajectory analysis

20 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Trajectory shape analysis Trajectory shape analysis for abnormal behavior recognition in video surveillance. Different context than VLDB: few and noisy data, high degree of variability, tracking errors Trajectory Shape similarity; invariant to space shifts Not only space-based or time-based similarity

21 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Trajectory Shape Analysis by angles Sequence of 2D spatial coordinates Sequence of 1D angles Advantages of using angles: more compact representation invariant to spatial translations (both local and global), thus describing trajectory shape

22 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Imagelab Proposal 1. Trajectory description with angle sequence 2. Statistical representation with a Mixture of Von Mises Distributions (MovM) 3. Coding with a sequence of selected vM pdf identifiers 4. Code Alignment 5. Clustering with k-medoids A. Prati, S. Calderara, R. Cucchiara, "Using Circular Statistics for Trajectory Analysis" in Proceedings of CVPR 2008 Definition of EM algorithm for MovM Using Dynamic programming Definition of Bhattacharyya distance fon vM and on-line EM

23 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Training set and on-line classification Clustering with Br distance Alignement Trajectory clusters repository MovM(T j ) EM for MoVM Trajectory repository Coding with MAP On-line EM for MoVM Coding with MAP Alignement Classification with Br distance Surveillance system Normal/ abnormal

24 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Von Mises distribution When the variables represent angles, Gaussians or MoGs are inappropriate. Example: two observations at 1° and 359°. Modeling these data with a univariate Gaussian distribution is incorrect. In fact, if we select the origin at 0° if we select the origin at 180° Von Mises distribution is more suitable to treat periodic variables, being circularly defined I 0 = modified zero-order Bessel function of the first kind

25 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Mixture of von Mises and Mixture of Gaussians (MoG) MovM: MoG:

26 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Modelling a single trajectory 1)A single trajectory is modeled as a sequence of angles: 2) A specifically defined EM algorithm is used:

27 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy EM for MovM distribution MovM: Likelihood of complete data set: Expected value of the log likelihood: E-step: estimate of the responsabilities:

28 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy EM for MovM distribution M-step: maximizing wrt : function zeros found by inverted numerically

29 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Mapping angles to symbols 2) A single trajectory is modeled as a sequence of angles and after having defined the MoVM as a sequence of symbols:

30 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Distance for sequences We transform a comparison between two sequences of either angles or coordinates in the comparison between two sequences of symbols, with each symbol corresponding to the proper probability distribution However, due to acquisition noise, uncertainty and spatial/temporal shifts, exact matching between sequences is unsuitable for computing similarity We use global alignment between two sequences, basing the distance as a cost of the best alignment of the symbols Dynamic programming techniques are used to speed up the process.

31 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Global alignment Global vs local alignment Using global alignment instead of local one is preferable because the former preserves both global and local shape characteristics Dynamic programming is used to reduce computational time to O (ni · nj), where ni and nj are the lengths of the two sequences.

32 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Inexact matching Since the symbols we are comparing correspond to pdf, match/mismatch should be proportional to the distance between the two corresponding pdfs Need to evaluate distance between two pdfs: Angular: Von Mises Distributions Bhattacharyya distance bw pdfs (closed form) Spatial: Gaussians Distributions Bhattacharyya distance bw pdfs ( )

33 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Sequence similarity where c B is the Bhattacharyya coefficient The best alignment is then converted in a distance and used for clustering and testing

34 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Comparison of alignment techniques When the sequences are characterized by different lengths, DTW tries to stretch the two sequences in order to find the optimal time warping path with the consequence of eventually adding additional matches. Global alignment (based on Needleman-Wunsch algorithm), on the other hand, simply adds gaps to align the sequences leading to the advantage of being more susceptible to slight time series changes by controlling the gap cost value

35 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Comparison of alignment techniques

36 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Clustering trajectories The distance is used to cluster the trajectories in the training set either according their shape or they location k-medoids algorithm: prototype of the cluster is the element that minimizes the sum of intra-class distances To compute the best number of k clusters, iterative k- medoids: initialization: i = 0, k(0) = Nt (cardinality training set); each trajectory is chosen as medoid) of the cluster Step 1: Run k-medoids algorithm with k(i) clusters Step 2: If there are two medoids with a similarity greater than a threshold Th, merge them and set k(i+1) = k(i)1. Increment i and go back to step 1.

37 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Experimental Result We report results on a corpus of 3000 trajectories with an average length of 100 points We compare our method with the baseline off-line time sequence comparison method of [Keog02] E. Keogh., Exact indexing of dynamic time warping, in 28th International Conference on Very Large Data Bases. Hong Kong, 2002, pp. 406–417 MethodClassification Accuracy Normal Abnormal Accuracy Online VM + GA96%97% Gaussian + Online GA 93%97% [Keog02] on complete trajectory 85%87%

38 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Comparison between VS and VLDB approaches

39 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Comparison between VS and VLDB approaches Results on synthetic dataset

40 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Comparison between VS and VLDB approaches Results on real dataset

41 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Adding the speed Pure trajectory shape is not sufficiently always discriminative in surveillance scenarios: the same path covered by a walk or by a run has a different meaning in terms of behavior Add the speed to the shape description to provide a more complete analysis of the trajectory.

42 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Trajectory encoding For each couple of subsequent point the angle θ and the velocity vector module ρ are computed For each couple of parameters (θ i, ρ i ) the encoding is performed using a polar scheme Velocity module is used to choose the ring and the direction is used to choose the sector

43 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Alignment score for trajectory comparison After the polar encoding a trajectory T i is then represented as a sequence of literals S={s i,1,s i,2,s i,3 …} We define a suitable score to compare people trajectories given two simbols s p,i and s q,j and the corresponding codes c a1,b1 and c a2,b2 The matching score λ i,j is finally normalized to 1 and the similarity metric ξ i,j is computed

44 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Experiments We log for training 88 trajectories from the multicamera system at our campus during ordinary working days We collect 121 trajectories for testing purposes being labeled manually by an expert as belonging to one of the 12 clusters previously computed The classification rate is 74%. Most of errors are due to two main factors: First: lack of data in the training set Second: inherent difficulties for the expert to answer the question Which is the most similar trajectory in the direction and the velocity domain?

45 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Experiments Error example: S. Calderara, R. Cucchiara, A. Prati, "A Dynamic Programming Technique for Classifying Trajectories" in Proceedings of IEEE International Conference on Image Analysis and Processing (IEEE ICIAP 2007), Modena, Italy, pp , Sept , 2007

46 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Trajectory modeling Use of semi-directional statistics to jointly model linear (speed) and circular (direction) data Estimation of precision m in Von Mises pdf is troublesome Using a approximated wrapped Gaussian pdf is preferable: Similar treatment of its linear counterpart a linear approximation of the variance parameter even for circular variables: Gaussian MLE to compute the joint multivariate covariance matrix 46

47 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Checking independence since directions and speed are dependent :

48 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy State of the Art approaches H. Ding, G. Trajcevski, P. Scheuermann, X. Wang, and E. J. Keogh,Querying and mining of time series data: experimental comparison of representations and distance measures, Proceedings of the VLDB Endowment, vol. 1, no. 2, pp. 1542–1552, N. Piotto, N. Conci, and F. De Natale. Syntactic matching of trajectories for ambient intelligence applications. IEEE Transactions on Multimedia, 11(7):1266–1275, Nov We choose to test our MoAWLG method against two state of the art approaches: Point-to-point, Complete, Original: (Ding- Keogh08) (same as before, but with also speed) Point-to-point, Selected, Transformed: (Piotto09)

49 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy (Piotto09) Point-to-point Selected Quantized The method proposed in (Piotto 09) perform the comparison among selected quantize representations of the original position-speed dataspace. Characteristic points of the sequences (breakpoints) are extracted: Temporal Breakpoints: consecutive points in a small area are represented by a single point associated with the time interval the objects stays in its position Spatial Breakpoints: sudden(a) or slow curvature changes(b) are selected as representative points of the trajectory.

50 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy (Piotto09) Point-to-point Selected Quantized (2) Once the breakpoints B are computed two consecutive breakpoints identifies a segment. Every segment is then associated to a symbol Where :

51 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy (Piotto09) Point-to-point Selected Quantized (3) Every Symbols values are quantized and associated to literals: Directions are quantized not uniformly Speed and time are quantized in fixed intervals Symbols sequences are aligned using Global Alignment separately for every dimension (direction,speed,time) and the final similarity score is a weighted sum of partial scores.

52 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Experimental comparison We compare our AWLG method with the approaches in (Ding08) and (Piotto09) on a dataset of about 500 trajectories manually ground truthed and divided in clusters We perform 4 tests: T1 and T2: ordinary days acquired trajectories T3: Actor played straight trajectories T4: T3 Trajectories at different speeds.

53 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Experimental comparison Clustering accuracy was measured using the same K-medoids based clustering on distance matrices computed with the different methods described Test IDNumber of Trajectories (Ding08)(Piotto09)Our Approach T114078%73% 95% T210880%87% 99% T314594%86% 96% T410090%80% 97%

54 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Conclusions Trajectory analysis is one of the most powerful task to compare movements of people many and many different proposals for large datasets of long trajectories typical data series comparisons point to point and complete could be preferable With smaller and noisy dataset statistical methods could be the best ones - With MoG for spatial representation - With MoVM for shape representation only - With MoAWLG for shape and speed representation

55 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy Multiple camera and distributed tracking Multi-camera tracking with camera with overlapping FOVs: Use calibration and 3D geometry Improve with Probabilistic Association Distributed Tracking with camera without overlapping FOVs: Search for similarity Content based retrieval methods Global descriptors: Histograms texture.Medionis circular histograms, Mixture of gaussians….. S. Calderara, R.Cucchiara, A. Prati Multimedia Surveillance: Content based Retrieval with Multicamera People Tracking Proc of VSSN 2006

56 ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy For any other information Rita Cucchiara Dipartimento di Ingegenria dellInformazione Thanks to Imagelab Andrea Prati, Roberto Vezzani, Costantino Grana, Simone Calderara, Giovanni Gualdi, Paolo Piccinini, Paolo Santinelli, Daniele Borghesani, Davide Baltieri, Sara Chiossi, Rudy Melli, Emanuele Perini, Giuliano Pistoni..


Download ppt "ImageLab Modena Rita Cucchiara - Università di Modena e Reggio Emilia, Italy People Movement analysis: trajectories Behavior analysis is a crucial tool."

Similar presentations


Ads by Google