Presentation is loading. Please wait.

Presentation is loading. Please wait.

Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca.

Similar presentations


Presentation on theme: "Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca."— Presentation transcript:

1 Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca

2 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 2 Relations to improve tracking

3 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 3 Complex activity recognition Y.Ke, R.Sukthankar, M.Hebert; Event Detection in Crowed Videos

4 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 4 Objectives Goals: 1.To model relations and 2.To maintain beliefs over particular relations between objects In order to simultaneously: Improve tracking with informed predictions and Identify complex activities based on observations and prior knowledge

5 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 5 Relational Domain Relational Domain: set of objects characterized by attributes 1 and with relations 1 between them Car Id color position(t) velocity(t) direction(t) DecreasingVelocity(t) A SameDirection(t) distance(t) Before(t) Car B Id color position(t) velocity(t) direction(t) DecreasingVelocity(t) SameDirection(t) distance(t) Before(t) 1 Attributes and relations are predicate in FOL.

6 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 6 Relational State The State of a Relational Domain is the set of the predicates that are true in the Domain. Relational state State of attributes State of relations

7 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 7 Relational Bayesian Networks: Uncertainty in a Relational Domain Relational (Dynamic) Bayesian Networks Syntax RBN: –a set of nodes, one for each variable –a directed, acyclic graph –a conditional distribution for each node given its parents This distribution must take into account the actual complexity of the nodes! Syntax RBN: –a set of nodes, one for each predicate –a directed, graph –a conditional distribution for each node given its parents

8 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 8 Dynamics The State of a Relational Domain is the set of the predicates that are true in the Domain. State evolves with time We extend a RBN to a RDBN as we are used to extend a BN to a DBN.

9 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 9 Inference Markov assumption and Conditional independence of data on state. bel(s t ) = ® p(z t |s t ) s p(s t |s t-1 )bel(s t-1 )ds t-1 Bayesian Filter The problem of computing: bel(s t ) = p(s t |z 1:t )

10 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 10 Inference Relations in the State result in correlating the State of different objects between them p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1

11 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 11 Sensor model (1 st assumption) part of the state relative to relations, s r, not directly observable p(z t |s t ) = p(z t |s a t ) observation z t independent by the relations between objects. Intuitively: Travelling Together vs Being Close

12 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 12 Transition model: a trick p(s t |s t-1 ) = p(s a t,s r t |s a t-1, s r t-1 ) S a t-1 S r t-1 SatSat SrtSrt Intuitive

13 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 13 p(s a t,s r t |s a t-1,s r t-1 )= But s r t independent by s a t-1 given s r t-1 and s a t p(s a t,s r t |s a t-1,s r t-1 ) = p(s a t |s a t-1,s r t-1 ) p(s r t |s r t-1, s a t ) bel(s t ) = p(s t |z 1:t ) = p(s a t,s r t |z 1:t ) bel(s t )=αp(z t |s a t,s r t ) s p(s a t,s r t |s a t-1,s r t-1 )bel(s t-1 )ds t-1 p(z t |s a t,s r t ) = p(z t |s a t ) Relational Inference p(s a t |s a t-1,s r t-1 ) p(s r t |s a t-1,s r t-1, s a t ) Transition model (2 nd assumption)

14 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 14 * It is a technique that implements a recursive Bayesian Filter through a Monte Carlo simulation. The key idea is to represent the posterior pdf as a set of samples (particles) paired with weights and to filter the mesurament based on these weights.. Particle Filtering* (general case)

15 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 15 Relational Particle Filter

16 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 16 RPF: extraction X a t,(m) X r t,(m) X a t,(m) ~ p(x a t,(m) |s a t-1,s r t-1 ) X a t,(m) ~ p(x r t,(m) |s a t = x a t,(m),s r t-1 ) X r t,(m)

17 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 17 RPF: weighting The consistency of the probability function ensures the convergence of the algorithm. X a t,(m) X r t,(m) Weight ( ) ~p(z t |x a t ) The weighting step is done according to the attributes part of each particle only, the relational part follows.

18 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 18 Experiments: FOPT

19 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 19 Experiments: Transition Model If relation true If relation false

20 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 20 Experiments: Results

21 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 21 Further experiments Data: 15 simulated objects. From each cell, an object can jump to one of the n next cells where n depends by the cell. Objects can move together. If traveling together, two (or more) objects will always be in cells from which it is possible for one to reach the other or vice-versa. If traveling together, two objects will behave similarly (i.e. if one turns left, the other will follow).

22 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 22 Tracking AND activity Recognition X a t,(m) X r t,(m) X a t,(m) X r t,(m) X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) 1° step of sampling: prediction of the state of attributes X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) X a {t,(m)} X o {t,(m)} X r t+1,(m) 2° step of sampling: prediction of the state of relations Or activity prediction

23 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 23 step 12 step 24 True Positive Rate False Positive Rate The worst (time step 24) and the best (time step 12) ROC curve for the relation recognition task. Further Results 01 1

24 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 24 PF: 4.6500 2.2333 3.7333 2.7667 … RPF: 4.6000 2.4667 1.3333 2.1000 … PF : 4.7000 3.6667 5.6667 2.6000 … RPF: 4.6000 3.5333 5.2667 2.5333 … Further Results (cont.) Tracking error (distance) for each of the 15 objects. Comparable behaviour of the errors BUT for related objects RPF trackes always better than PF. PF : 4.6667 4.6667 3.8333 1.9333 … RPF: 4.7667 2.7667 3.5333 1.5333 … PF: 2.0667 5.9000 1.6000 RPF: 2.0333 5.8333 2.2333

25 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 25 To conclude... Modeling Relations dynamically: –To improve multi target tracking –To recognize complex activities Inference in Dynamic Relational Domain – In theory complex BUT – Simplified by smart decomposition of the transition model non-relational sensor model Showed promising results

26 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 26 Related works Complex tracking tasks: –Heuristics M.Isard and J. MacCormick BraMBLe, A Bayesian Multi-Blob Tracker.

27 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 27 Related works Complex tracking tasks: –Heuristics –Mixed-States models Complex activity recognition: –Stochastic grammar Free Y.A.Ivanov and A.F.Bobick Recognition of Visual Activities and Interactions by Stochastic Parsing

28 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 28 Related works Complex tracking tasks: –Heuristics, –Mixed-States models Complex activity recognition: –Stochastic grammar Free, –First Order Logic S. Tran and L. Davis, Visual Event Modeling and Recognition using Markov Logic Networks

29 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 29 p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1 ~ Transition model Sensor model Inference Relations in the State result in correlating the State of different objects between them p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1

30 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 30 Conditional Probability Distribution FOPT: a Probabilistic Tree whose nodes are FOL formulas CPD relation t (x,y): relation t-1 (x,y) p(relation t (x,y)) x t, y t CPD y t : x, relation t-1 (x,y) p(y t |y t-1 ) T F p(y t |y t-1 ) p(x t |x t-1,y t-1,r t-1 )

31 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 31 Tracking AND activity Recognition X a t,(m) X r t,(m) X a t,(m) X r t,(m) X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) 1° step of sampling: prediction of the state of attributes X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) X a {t,(m)} X o {t,(m)} X r t+1,(m) 2° step of sampling: prediction of the state of relations Or activity prediction

32 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 32 BN: the Alarm example

33 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 33 Thanks to Mark Chavira A large BN

34 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 34 The Alarm Relational Domain Relational Domain contains a set of objects with relations between them Objects e.g.: Relation neighbor alarm burglar toCall (the howner of the house) toHear (the alarm) neighbors attributes: capacity of hearing, attention,... alarms attributes: its volume, its sensibility,...

35 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 35 Alarm RBN: Alarm.Volume NeighborCalls Earthquacke Neigh.DegOfDef Neigh.NoiseAround

36 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 36 Closing the parenthesis... Syntax RBN: –a set of nodes, one for each variable –a directed, acyclic graph –a conditional distribution for each node given its parents This distribution must take into account the actual complexity of the nodes! Syntax RBN: –a set of nodes, one for each predicate –a directed, graph –a conditional distribution for each node given its parents,

37 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 37 Related works Complex tracking tasks: –Heuristics –Mixed-States models M.Isard and A.Blake A mixed-state condensation tracker with automatic model switching

38 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 38 Canadian Harbor: rendezvous Same speed

39 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 39 Canadian Harbor: Avoidance Constant speed

40 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 40 Exp: attributestransition

41 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 41 Exp: relationstransition

42 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept. 28 - Oct 2, 2009 42 Exp: Results methodTP ratioTN ratio Mean Tracking Error (km) RPF0.45450.72351.8379 PF3.3906 random0.44440.4841


Download ppt "Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca."

Similar presentations


Ads by Google