Computational Movement Analysis Lecture 4: Movement patterns Joachim Gudmundsson

Movement patterns Much of the early work on movement analysis focussed on finding movement patterns. For example: Flocks (group of entities moving close together) Swarm Convoys Heards Following (is an entity following another entity) Leadership Single file Popular places (place visited by many) …

Movement patterns: Challenge Find definitions of movement patterns that are: 1.Useful and 2. “Computable”

Movement patterns: Notation n – number of entities (trajectories)  - maximum complexity of a trajectory

Movement patterns: Groups

t1t1 t2t2 t3t3 t4t4 Kalnis, Mamoulis & Bakiras, 2005 G. & van Kreveld, 2006 Benkert, G., Hubner & Wolle, 2006 Jensen, Lin & Ooi, 2007 Al-Naymat, Chawla & G., 2007 Vieria, Bakalov & Tsotras, 2009 Jeung, Yiu, Zhou, Jensen & Shen, 2008 Jeung, Shen & Zhou, Movement patterns: Groups

Time = m = 3 m – flock size Movement patterns: Groups

flock meet fixed subset variable subset examples for m = 3 Movement patterns: Groups

Movement patterns: Convoy

X – set of entities m – convoy size  – distance threshold Define a group: x,y  X are directly connected if the  -disks of x and y intersect. x 1 and x k are  -connected if there is a sequence x 1,x 2,…, x k of entities such that  i, x i and x i+1 are directly connected. x y  x1x1 x2x2 x3x3 x4x4 x5x5

[Jeung, Yiu, Zhou, Jensen & Shen, 2008] [Jeung, Shen & Zhou, 2008] A group of entities form a convoy if every pair of entities are  -connected. Movement patterns: Convoy

Movement patterns: Leadership & Followers

front A leader? - Should not follow anyone else! - Is followed by at least m other entities. - For a certain duration. Movement patterns: Leadership & Followers

A leader? - Should not follow anyone else! - Is followed by at least m other entities. - For a certain duration. Many different settings. Running time ~O(n 2  log n  ) Movement patterns: Leadership & Followers [Andersson et al. 2007]

Movement patterns: Popular places

σ A region is a popular place if at least m entities visit it. σ is a popular place for m  5 [Benkert, Djordjevic, G. & Wolle 2007] Movement patterns: Popular places

Continuous model: maximum number of visitorsO(  2 n 2 )  (  2 n 2 ) Discrete model: maximum number of visitorsO(  n log  n)  (  n log  n)

Movement patterns: Single File

Single file: Intuitively easy to define Hard to define formally!

Movement patterns: Single File Single file: Intuitively easy to define Hard to define formally!

Movement patterns: Single File Single file: Intuitively easy to define Hard to define formally!

Towards a Formal Definition? We say that the entities x 1, …, x m are moving in single file for a given time interval if during this time each entity x j+1 is following behind entity x j for j = 1, …,m-1.

Towards a Formal Definition?  Time t’  [t+  min,t+  max ] Time t We say that the entities x 1, …, x m are moving in single file for a given time interval if during this time each entity x j+1 is following behind entity x j for j = 1, …,m-1. Following behind?

Define Following Behind Let x 1 be an entity with parameterized trajectory f 1 over the time interval [s 1,t 1 ] and let x 2 be an entity with parameterized trajectory f 2 over the time interval [s 2,t 2 ], where s 2  [s 1 +  min, s 1 +  max ] and t 2  [t 1 +  min, t 1 +  max ]. Entity x 2 is following behind x 1 in [s 1,t 1 ] if there exists a continuous, bijective function  : [s 1,t 1 ]  [s 2,t 2 ] such that  (s 1 )=s 2 and  t  [s 1,t 1 ]:  (t)  [t -  max,t-  min ]  d(f 1 (  (t)),f 2 (t))  . f1f1 f2f2

Define Following Behind f1f1 f2f2 Let x 1 be an entity with parameterized trajectory f 1 over the time interval [s 1,t 1 ] and let x 2 be an entity with parameterized trajectory f 2 over the time interval [s 2,t 2 ], where s 2  [s 1 +  min, s 1 +  max ] and t 2  [t 1 +  min, t 1 +  max ]. Entity x 2 is following behind x 1 in [s 1,t 1 ] if there exists a continuous, bijective function  : [s 1,t 1 ]  [s 2,t 2 ] such that  (s 1 )=s 2 and  t  [s 1,t 1 ]:  (t)  [t -  max,t-  min ]  d(f 1 (  (t)),f 2 (t))  .

Free Space Diagram If there exists a monotone path in the free-space diagram from (0, 0) to (p, q) which is monotone in both coordinates, then curves P and Q have Fréchet distance less than or equal to ε (0,0) (p,q)

Free Space Diagram and Following Behind The time delay means that the path will be restricted to a “diagonal” strip! F C (f 1,f 2 ) = {(s,t) | d(f 1 (s),f 2 (t))    t-s  [  min,  max ]}.  min  max Running time: O(  +k) where k is the complexity of the diagonal strip

Algorithm: Single file One can determine in O(  k 2 ) time during which time intervals one trajectory is following behind the other. If the order between the entities is specified then compute the free space diagram between every pair of consecutive entities: Time: O(n  k 2 )

No Order? If no order then compute the free space diagram between every pair of entities: Time: O(n 2  k 2 ) x1x1 x2x2 x3x3 x4x4 ([3,8],[14,21]) x1x1 x2x2 x3x3 x4x4 [Buchin et al. 2008]

Trajectory grouping How to define and compute the structure of groups of moving entities, including merging and splitting? [Buchin, Buchin, van Kreveld, Speckmann and Staals 2013]

Grouping We do not want this to be considered: A merge of two groups into one, followed by a split of a group into two

Grouping We probably want this to be considered: A merge of two groups into one, followed by a split of a group into two

Grouping: definitions X – set of entities  - complexity of each input trajectory Define a group: x,y  X are directly connected if the  -disks of x and y intersect. x 1 and x k are  -connected if there is a sequence x 1,x 2,…, x k of entities such that  i, x i and x i+1 are directly connected. x y  x1x1 x2x2 x3x3 x4x4 x5x5

Grouping: definitions A set S of entities is  -connected if all entities in S are pairwise  -connected.  The  -disks of the entities in S form a connected component.

Grouping: definitions C(t) – the set of connected components at time t that forms a partition of X. C(t) consists of 5 components Time: t

Grouping: definitions What is a group? Three criteria for a group: - big enough (size m) - close enough (  -connected) - long enough (duration δ) Only maximal groups are relevant (maximal in group size, starting time or ending time) m=4 

Grouping: definitions What is a group? Three criteria for a group: - big enough (size m) - close enough (  -connected) - long enough (duration δ) Only maximal groups are relevant (maximal in group size, starting time or ending time) m=4 duration>δ

Grouping: definitions Note that an entity can be in several maximal groups at the same time!

Grouping:definitions Note that an entity can be in several maximal groups at the same time!

Grouping:definitions Note that an entity can be in several maximal groups at the same time!

Grouping: definitions Note that an entity can be in several maximal groups at the same time! t=0 t=1 t=2 t=3 t=[0,2] : red/blue t=[1,2] : red/blue/green t=[1,3] : red/green At time t=2 red is in 3 groups

Trajectory grouping structure

purple red, blue, green blue, green red green purple, green red blue red, blue t=0 t=1t=4 t=5 t=8 t=10 t=0 t=10

Grouping Questions: 1.Number of maximal groups? 2.How can we effectively compute all the maximal groups?

Grouping Idea: Consider the motion of the  -disks of the entities over time.

Grouping Idea: Consider the motion of the  -disks of the entities over time.  Union of n tubes. Denote this manifold by M. Each tube consists of  skewed cylinders with horizontal radius . We can see it as tracing the  -disk of an entity over its trajectory.

Grouping We are interested in horizontal cross-sections, and the evolution of the connected components. This is captured by the so-called Reeb graphs.

Reeb graph Reeb graph, with maximal groups associated to edges captures the changes in connectivity of a process, using a graph – Edges are connected components – Vertices define changes in connected components (events)

Reeb graph Four types of vertices: start vertex (time t 0 ) – in-degree 0, out-degree 1 end vertex (time t  ) – in-degree 1, out-degree 0 merge vertex - in-degree 2, out-degree 1 split vertex - in-degree 1, out-degree 2

Complexity of the Reeb graph Observation: Reeb graph of M has  (  n 2 ) vertices and edges. Proof: Construction for one time step [t i,t i+1 ].

Complexity of the Reeb graph Observation: Reeb graph of M has  (  n 2 ) vertices and edges. Proof: Construction for one time step [t i,t i+1 ].

Complexity of the Reeb graph Observation: Reeb graph of M has  (  n 2 ) vertices and edges. Proof: Construction for one time step [t i,t i+1 ].  n/2  n/2 =  (n 2 ) components/ time step. □

Complexity of the Reeb graph Observation: The Reeb graph of M has O(  n 2 ) vertices. Proof: Consider two entities during one time interval [t i,t i+1 ]. This interval can generate at most two vertices in the the Reeb graph.

Complexity of the Reeb graph Observation: The Reeb graph of M has O(  n 2 ) vertices. Proof: Consider two entities during one time interval [t i,t i+1 ]. This interval can generate at most two vertices in the the Reeb graph.  O(n 2 ) components/time step. □

Computing the Reeb graph Compute all events: For every pair of skewed cylinders compute the time of intersection. Time: O(  n 2 ) Sort the events with respect to time: Time: O(  n 2 log  n)

Computing the Reeb graph Compute & sort all events: Time: O(  n 2 log  n) Initialize graph: Time: O(  n 2 ) Handle events by increasing time: Time: O(  n 2 log n) Using a dynamic ST-trees [Sleator & Tarjan’83] [Parsa‘12]

Computing the Reeb graph Theorem: The Reeb graph of M can be computed in time O(  n 2 log  n).

Number of maximal groups Each entity follows a directed path in the Reeb graph starting at a start vertex and ending at an end vertex. purple red, blue, green blue, green red green purple, green red blue red, blue t=0 t=1t=4 t=5 t=8 t=10 t=0 t=10

Number of maximal groups Theorem: There are O(  n 3 ) maximal groups. Proof: There are O(  n 2 ) vertices in the Reeb graph. Prove that O(n) maximal groups can start at a start or merge vertex. purple red, blue, green blue, green red green purple, green red blue red, blue t=0 t=1t=4 t=5 t=8 t=10 t=0 t=10

Number of maximal groups Consider a merge vertex v. Assume sets S and T merge at v. S T v

Number of maximal groups Consider a merge vertex v. Assume sets S and T merge at v. Let p x denote the path of entity x  S  T starting at v. S T pxpx v

Number of maximal groups Consider a merge vertex v. Assume sets S and T merge at v. Let p x denote the path of entity x  S  T starting at v. Consider the union R’ of all such paths from entities in S  T. R’ is a directed acyclic subgraph of the Reeb graph. S T v

Number of maximal groups Consider a merge vertex v. Assume sets S and T merge at v. Let p x denote the path of entity x  S  T starting at v. Consider the union R’ of all such paths from entities in S  T. R’ is a directed acyclic subgraph of the Reeb graph. “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. S T v pxpx pypy R’R’

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. v pxpx pypy v pxpx pypy

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. v pxpx v pxpx pypy pypy

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. v v

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. vv

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. A maximal group can only end at an end or split vertex. v

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. A maximal group can only end at an end or split vertex. Number of leaves = |S|+|T|  n v

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. A maximal group can only end at an end or split vertex. Number of leaves = |S|+|T|  n Number of split vertices? Every vertex has degree at most 3.  O(n) split vertices. v

Number of maximal groups “Unravel” R’: if p x and p y split and then merge at a vertex u then duplicate the paths starting at u. A maximal group can only end at an end or split vertex. Number of leaves = |S|+|T|  n Number of split vertices? Every vertex has degree at most 3.  O(n) split vertices. There can be at most one maximal group starting at v and ending at a split vertex w.  O(n) maximal groups starting at v. v

Number of maximal groups Theorem: There are O(  n 3 ) maximal groups. Proof: There are O(  n 2 ) vertices in the Reeb graph. There are O(n) maximal groups can start at a start or merge vertex. □ v

Computing the maximal groups Input: Given a set of trajectories and three parameters m,  and . 1.Compute the Reeb graph using distance . Time: O(  n 2 log  n) 2.Process the vertices in time-order, maintaining maximal groups t0t0 tGtG (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) t1t1 t2t2 (1,t 0 ) (2,t 0 ) (12,t 1 ) (3,t 0 ) (4,t 0 ) (34,t 1 ) t3t3 t4t4 (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) 1,3 2,4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 )

Computing the maximal groups 2.Maintain maximal groups Each edge e=(u,v) is labelled with a set of maximal groups G e. Note: A group G becomes maximal at a vertex. t0t0 tGtG (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) t1t1 t2t2 (1,t 0 ) (2,t 0 ) (12,t 1 ) (3,t 0 ) (4,t 0 ) (34,t 1 ) t3t3 t4t4 (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) 1,3 2,4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 )

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. Note that a group G becomes maximal at a vertex v. a.Start vertex: Set G e = {(C e,t v =t 0 )} v G e ={(C e,t v =t 0 )} u

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. Note that a group G becomes maximal at a vertex v. a.Start vertex: Set G e = {(C e,t v =t 0 )} b.Merge vertex: Propagate the maximal groups from “children” to e and add (C e,t v ). v u Ge1Ge1 Ge2Ge2 G e ={G e 1  G e 2  (C e,t v )}

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. Note that a group G becomes maximal at a vertex v. a.Start vertex: Set G e = {(C e,t v =t 0 )} b.Merge vertex: Propagate the maximal groups from “children” to e and add (C e,t v ). c.Split vertex: v Ge1Ge1 GeGe Ge2Ge2

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. Note that a group G becomes maximal at a vertex v. a.Start vertex: Set G e = {(C e,t v =t 0 )} b.Merge vertex: Propagate the maximal groups from “children” to e and add (C e,t v ). c.Split vertex: - A group may end at v (if group splits) - A group may start at v on either e 1 or e 2 - A group may continue on e 1 or e 2 v Ge1Ge1 GeGe Ge2Ge2 Question: How can we compute these groups efficiently?

Store maximal groups e Observation: Assume S, T in G e. S starting at t S and T starting at t T. tStS tTtT G e = {S,T,…}

Store maximal groups e Observation: Assume S, T in G e. S starting at t S and T starting at t T.  G  S  T in G e with starting time t G  max{t S,t T }. tStS tTtT tGtG G e = {S,T,G,…}

Store maximal groups e Observation: Assume S, T in G e. S starting at t S and T starting at t T.  G  S  T in G e with starting time t G  max{t S,t T }. If S  T  then S  T or T  S. tStS tTtT tGtG G e = {S,T,G,…}  S and T are disjoint, S  T or T  S

Represent G e by a tree T e. Each node v in T e represents a group G v in G e. The children of v are the largest subgroups of of G v. t0t0 tGtG (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) t1t1 t2t2 (1,t 0 ) (2,t 0 ) (12,t 1 ) (3,t 0 ) (4,t 0 ) (34,t 1 ) t3t3 t4t4 (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) 1,3 2,4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 ) Store maximal groups

Represent G e by a tree T e. Each node v in T e represents a group G v in G e. The children of v are the largest subgroups of of G v. t0t (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) t1t1 t2t2 (1,t 0 ) (2,t 0 ) (12,t 1 ) (3,t 0 ) (4,t 0 ) (34,t 1 ) t3t3 t4t4 1,3 2,4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 ) (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) {1,2,3,4} {1,2}{3,4} {1}{2}{3}{4} Store maximal groups

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. a.Start vertex: Set G e = {(C e,t v =t 0 )} b.Merge vertex: Propagate the maximal groups from “children” to e and add (C e,t v ). c.Split vertex: - A group may end at v (if group splits) - A group may start at v on either e 1 or e 2 - A group may continue on e 1 or e 2 (1234,t 2 ) (12,t 1 ) (34,t 1 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) t2t2 t3t3 t4t4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 ) (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) 1,3 2,4

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. a.Start vertex: Set G e = {(C e,t v =t 0 )} b.Merge vertex: Propagate the maximal groups from “children” to e and add (C e,t v ). c.Split vertex: - A group may end at v (if group splits) - A group may start at v on either e 1 or e 2 - A group may continue on e 1 or e 2 (1234,t 2 ) (12,t 1 ) (34,t 1 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) t2t2 t3t3 t4t4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 ) (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) 1,3 2,4

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. a.Start vertex: Set G e = {(C e,t v =t 0 )} b.Merge vertex: Propagate the maximal groups from “children” to e and add (C e,t v ). c.Split vertex: - A group may end at v (if group splits) - A group may start at v on either e 1 or e 2 - A group may continue on e 1 or e 2 t2t2 t3t3 t4t4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 ) (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) (1234,t 2 ) (12,t 1 ) (34,t 1 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) 1,3 2,4

Computing the maximal groups 2.Annotate its edges and vertices Each edge e=(u,v) is labelled with a set of maximal groups G e. a.Start vertex: Set G e = {(C e,t v =t 0 )} b.Merge vertex: Propagate the maximal groups from “children” to e and add (C e,t v ). c.Split vertex: - A group may end at v (if group splits) - A group may start at v on either e 1 or e 2 - A group may continue on e 1 or e 2 t2t2 t3t3 t4t4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 ) (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) (1234,t 2 ) (12,t 1 ) (34,t 1 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) (13,t 3 ) (24,t 3 ) 1,3 2,4

Computing the maximal groups Analysis Start vertex: O(1) Merge vertex: O(1) Split vertex: O(|T e |) and |T e |=O(n) Total time: #vertices in the Reeb graph  O(n) = O(  n 3 ) + the total output size t2t2 t3t3 t4t4 (1,t 0 ) (3,t 0 ) (13,t 2 ) (2,t 0 ) (4,t 0 ) (24,t 2 ) (1,t 0 )(2,t 0 ) (3,t 0 )(4,t 0 ) (12,t 1 )(34,t 1 ) (1234,t 2 ) (12,t 1 ) (34,t 1 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) (1,t 0 ) (2,t 0 ) (3,t 0 ) (4,t 0 ) (13,t 3 ) (24,t 3 ) 1,3 2,4

Changing group size and duration minimum group size 3 For illustration: x-coordinate is time

smaller minimum group size (2) larger minimum group size (5)

larger minimum duration

smaller minimum duration

Robustness If a group of 6 entities has 1 entity leaving very briefly, should we really see this as - two maximal groups of size 6, and - one maximal group of size 5 ? 6 entities 5 entities

Robust grouping structure

Modify Reeb graph › at most O(  n 3 ) changes in the Reeb graph › compute robust maximal groups in O(  n 3 log n) time (plus output size) Passing encounter Collapse encounter

Summary Grouping structures [Buchin et al.’13] - Model for the grouping structure of moving entities - Algorithms for computing the grouping structure Many different movement patterns Moving groups Convoys Following/Leadership Single file Popular places (place visited by many)

Open Problem Interesting movement patterns? Query versions? Automatically set parameters?

References K. Buchin, M. Buchin, M. van Kreveld, B. Speckmann and F. Staals. Trajectory Grouping Structure. WADS, H. Jeung, M. L. Yiu, X. Zhou, C. S. Jensen and H. T. Shen. Discovery of Convoys in Trajectory Databases. PVLDB, K. Buchin, M. Buchin and J. Gudmundsson. Constrained Free Space Diagrams: a Tool for Trajectory Analysis, International Journal of Geographical Information Science, M. Andersson, J. Gudmundsson, P. Laube and T. Wolle. Reporting leaders and followers among trajectories of moving point objects. GeoInformatica, 2008.