Presentation is loading. Please wait.

Presentation is loading. Please wait.

Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca.

Similar presentations


Presentation on theme: "Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca."— Presentation transcript:

1 Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca

2 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Relations to improve tracking

3 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Complex activity recognition Y.Ke, R.Sukthankar, M.Hebert; Event Detection in Crowed Videos

4 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Objectives Goals: 1.To model relations and 2.To maintain beliefs over particular relations between objects In order to simultaneously: Improve tracking with informed predictions and Identify complex activities based on observations and prior knowledge

5 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Relational Domain Relational Domain: set of objects characterized by attributes 1 and with relations 1 between them Car Id color position(t) velocity(t) direction(t) DecreasingVelocity(t) A SameDirection(t) distance(t) Before(t) Car B Id color position(t) velocity(t) direction(t) DecreasingVelocity(t) SameDirection(t) distance(t) Before(t) 1 Attributes and relations are predicate in FOL.

6 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Relational State The State of a Relational Domain is the set of the predicates that are true in the Domain. Relational state State of attributes State of relations

7 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Relational Bayesian Networks: Uncertainty in a Relational Domain Relational (Dynamic) Bayesian Networks Syntax RBN: –a set of nodes, one for each variable –a directed, acyclic graph –a conditional distribution for each node given its parents This distribution must take into account the actual complexity of the nodes! Syntax RBN: –a set of nodes, one for each predicate –a directed, graph –a conditional distribution for each node given its parents

8 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Dynamics The State of a Relational Domain is the set of the predicates that are true in the Domain. State evolves with time We extend a RBN to a RDBN as we are used to extend a BN to a DBN.

9 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Inference Markov assumption and Conditional independence of data on state. bel(s t ) = ® p(z t |s t ) s p(s t |s t-1 )bel(s t-1 )ds t-1 Bayesian Filter The problem of computing: bel(s t ) = p(s t |z 1:t )

10 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Inference Relations in the State result in correlating the State of different objects between them p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1

11 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Sensor model (1 st assumption) part of the state relative to relations, s r, not directly observable p(z t |s t ) = p(z t |s a t ) observation z t independent by the relations between objects. Intuitively: Travelling Together vs Being Close

12 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Transition model: a trick p(s t |s t-1 ) = p(s a t,s r t |s a t-1, s r t-1 ) S a t-1 S r t-1 SatSat SrtSrt Intuitive

13 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, p(s a t,s r t |s a t-1,s r t-1 )= But s r t independent by s a t-1 given s r t-1 and s a t p(s a t,s r t |s a t-1,s r t-1 ) = p(s a t |s a t-1,s r t-1 ) p(s r t |s r t-1, s a t ) bel(s t ) = p(s t |z 1:t ) = p(s a t,s r t |z 1:t ) bel(s t )=αp(z t |s a t,s r t ) s p(s a t,s r t |s a t-1,s r t-1 )bel(s t-1 )ds t-1 p(z t |s a t,s r t ) = p(z t |s a t ) Relational Inference p(s a t |s a t-1,s r t-1 ) p(s r t |s a t-1,s r t-1, s a t ) Transition model (2 nd assumption)

14 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, * It is a technique that implements a recursive Bayesian Filter through a Monte Carlo simulation. The key idea is to represent the posterior pdf as a set of samples (particles) paired with weights and to filter the mesurament based on these weights.. Particle Filtering* (general case)

15 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Relational Particle Filter

16 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, RPF: extraction X a t,(m) X r t,(m) X a t,(m) ~ p(x a t,(m) |s a t-1,s r t-1 ) X a t,(m) ~ p(x r t,(m) |s a t = x a t,(m),s r t-1 ) X r t,(m)

17 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, RPF: weighting The consistency of the probability function ensures the convergence of the algorithm. X a t,(m) X r t,(m) Weight ( ) ~p(z t |x a t ) The weighting step is done according to the attributes part of each particle only, the relational part follows.

18 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Experiments: FOPT

19 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Experiments: Transition Model If relation true If relation false

20 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Experiments: Results

21 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Further experiments Data: 15 simulated objects. From each cell, an object can jump to one of the n next cells where n depends by the cell. Objects can move together. If traveling together, two (or more) objects will always be in cells from which it is possible for one to reach the other or vice-versa. If traveling together, two objects will behave similarly (i.e. if one turns left, the other will follow).

22 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Tracking AND activity Recognition X a t,(m) X r t,(m) X a t,(m) X r t,(m) X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) 1° step of sampling: prediction of the state of attributes X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) X a {t,(m)} X o {t,(m)} X r t+1,(m) 2° step of sampling: prediction of the state of relations Or activity prediction

23 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, step 12 step 24 True Positive Rate False Positive Rate The worst (time step 24) and the best (time step 12) ROC curve for the relation recognition task. Further Results 01 1

24 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, PF: … RPF: … PF : … RPF: … Further Results (cont.) Tracking error (distance) for each of the 15 objects. Comparable behaviour of the errors BUT for related objects RPF trackes always better than PF. PF : … RPF: … PF: RPF:

25 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, To conclude... Modeling Relations dynamically: –To improve multi target tracking –To recognize complex activities Inference in Dynamic Relational Domain – In theory complex BUT – Simplified by smart decomposition of the transition model non-relational sensor model Showed promising results

26 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Related works Complex tracking tasks: –Heuristics M.Isard and J. MacCormick BraMBLe, A Bayesian Multi-Blob Tracker.

27 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Related works Complex tracking tasks: –Heuristics –Mixed-States models Complex activity recognition: –Stochastic grammar Free Y.A.Ivanov and A.F.Bobick Recognition of Visual Activities and Interactions by Stochastic Parsing

28 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Related works Complex tracking tasks: –Heuristics, –Mixed-States models Complex activity recognition: –Stochastic grammar Free, –First Order Logic S. Tran and L. Davis, Visual Event Modeling and Recognition using Markov Logic Networks

29 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1 ~ Transition model Sensor model Inference Relations in the State result in correlating the State of different objects between them p(x t-1 |z 1:t-1 )p(x t |z 1:t-1 )p(x t |z 1:t ) Bel(x t-1 ) Bel(x t ) Transition model Sensor model t = t+1

30 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Conditional Probability Distribution FOPT: a Probabilistic Tree whose nodes are FOL formulas CPD relation t (x,y): relation t-1 (x,y) p(relation t (x,y)) x t, y t CPD y t : x, relation t-1 (x,y) p(y t |y t-1 ) T F p(y t |y t-1 ) p(x t |x t-1,y t-1,r t-1 )

31 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Tracking AND activity Recognition X a t,(m) X r t,(m) X a t,(m) X r t,(m) X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) 1° step of sampling: prediction of the state of attributes X a t,(m) X a {t,(m)} X o {t,(m)} X r t,(m) X a t+1,(m) X a {t,(m)} X o {t,(m)} X r t+1,(m) 2° step of sampling: prediction of the state of relations Or activity prediction

32 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, BN: the Alarm example

33 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Thanks to Mark Chavira A large BN

34 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, The Alarm Relational Domain Relational Domain contains a set of objects with relations between them Objects e.g.: Relation neighbor alarm burglar toCall (the howner of the house) toHear (the alarm) neighbors attributes: capacity of hearing, attention,... alarms attributes: its volume, its sensibility,...

35 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Alarm RBN: Alarm.Volume NeighborCalls Earthquacke Neigh.DegOfDef Neigh.NoiseAround

36 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Closing the parenthesis... Syntax RBN: –a set of nodes, one for each variable –a directed, acyclic graph –a conditional distribution for each node given its parents This distribution must take into account the actual complexity of the nodes! Syntax RBN: –a set of nodes, one for each predicate –a directed, graph –a conditional distribution for each node given its parents,

37 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Related works Complex tracking tasks: –Heuristics –Mixed-States models M.Isard and A.Blake A mixed-state condensation tracker with automatic model switching

38 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Canadian Harbor: rendezvous Same speed

39 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Canadian Harbor: Avoidance Constant speed

40 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Exp: attributestransition

41 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Exp: relationstransition

42 C.Manfredotti, E.Messina: RDBNs to improve MTT. Mercure Chateau Chartrons, Bordeaux, France, Sept Oct 2, Exp: Results methodTP ratioTN ratio Mean Tracking Error (km) RPF PF random


Download ppt "Relational Dynamic Bayesian Networks to improve Multi-Target Tracking. Cristina Manfredotti and Enza Messina DISCo, University of Milano-Bicocca."

Similar presentations


Ads by Google