1. Darryl MorrellStochastic Modeling Seminar1 Particle Filtering

2. Darryl MorrellStochastic Modeling Seminar2 Organization of Slides Part I (PF from Dynamic Bayes Net Perspective) Understand particle filtering as a likelihood monte carlo sampling method on DBNs –Review of likelihood sampling –Uses R&N Part II (PF from general filtering perspective –Uses the Arulampalam tutorial

3. Darryl MorrellStochastic Modeling Seminar3

4. Darryl MorrellStochastic Modeling Seminar4

5. Darryl MorrellStochastic Modeling Seminar5

6. Darryl MorrellStochastic Modeling Seminar6

7. Darryl MorrellStochastic Modeling Seminar7

8. Darryl MorrellStochastic Modeling Seminar8

9. Darryl MorrellStochastic Modeling Seminar9 Outcome:

10. Darryl MorrellStochastic Modeling Seminar10

11. Darryl MorrellStochastic Modeling Seminar11 Problem: may need many many samples if the required probability is very low..

12. Darryl MorrellStochastic Modeling Seminar12

13. Darryl MorrellStochastic Modeling Seminar13

14. Darryl MorrellStochastic Modeling Seminar14

15. Darryl MorrellStochastic Modeling Seminar15

16. Darryl MorrellStochastic Modeling Seminar16

17. Darryl MorrellStochastic Modeling Seminar17

18. Darryl MorrellStochastic Modeling Seminar18

19. Darryl MorrellStochastic Modeling Seminar19

20. Darryl MorrellStochastic Modeling Seminar20

21. Darryl MorrellStochastic Modeling Seminar21

22. Darryl MorrellStochastic Modeling Seminar22

23. Darryl MorrellStochastic Modeling Seminar23

24. Darryl MorrellStochastic Modeling Seminar24

25. Darryl MorrellStochastic Modeling Seminar25

26. Darryl MorrellStochastic Modeling Seminar26 Particle Filters: a Solution to Hard Problems in Navigation, Target Tracking, and Perception Darryl Morrell & Ya Xue Department of Electrical Engineering Arizona State University Portions of this work supported by AFOSR under award number F49620-00-1-0124

27. Darryl MorrellStochastic Modeling Seminar27 References for More Information M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for online nonlinear/non- Gaussian Bayesian tracking. IEEE Transactions on Signal Processing, 50(2):174-188, February 2002. –This is an excellent tutorial paper-read this first. A. Doucet, N. de Freitas, and N. Gordon, editors. Sequential Monte Carlo Methods in Practice. Springer- Verlag, 2001. –This is a broad ranging collection of articles that will introduce your to most of the important particle filter developments.

28. Darryl MorrellStochastic Modeling Seminar28 Introduction “The importance of Monte Carlo methods for inference in science and engineering problems has grown steadily over the past decade. This growth has largely been propelled by an explosive increase in accessible computing power. …it has become clear that Monte Carlo methods can significantly expand the class of problems that can be addressed practically.” (Introduction to Feb 2002 IEEE Transactions on Signal Processing special issue on Monte Carlo Methods)

29. Darryl MorrellStochastic Modeling Seminar29 Sequential Monte Carlo Techniques Sequential Monte Carlo techniques have been developed in a wide range of disciplines, and go under many names: –Bootstrap filtering –The condensation algorithm –Particle filtering –Interacting particle approximations –Survival of the fittest

30. Darryl MorrellStochastic Modeling Seminar30 Presentation Outline Applications of particle filters Fundamental concepts Anatomy of a simple particle filter Variations on the simple particle filter Pros and Cons of particle filters Application to configuration of a foveal sensor Conclusions

31. Darryl MorrellStochastic Modeling Seminar31 Applications of Particle Filters Particle filters have provided solutions to problems from many disciplines: –image processing and understanding –tracking complex objects (e.g. people) in video sequences –robot navigation –tracking and identifying complex military targets (e.g. vehicle convoys)

32. Darryl MorrellStochastic Modeling Seminar32 Some Specific Applications Terrain aided navigation Car positioning using map information Robot navigation Tracking of articulated targets using video Tracking of complex targets using distributed sensors.

33. Darryl MorrellStochastic Modeling Seminar33 Terrain Aided Navigation http://www.control.isy.liu.se/research/sensorfusion/ sensorfusion/sensorfusion.html Observations are measured ground clearance. Unknowns are aircraft position and velocity. The particle filter is needed because measured ground clearance does not uniquely determine position.

34. Darryl MorrellStochastic Modeling Seminar34 Car Positioning Using Map Info Gustafsson et al., “Particle Filters for Positioning, Navigation, and Tracking,” IEEE Transactions on SP, Feb 2002 Observations are yaw rate and speed information computed from wheel speed sensors. Vehicle position is unknown. The map provides constraints on the vehicle position.

35. Darryl MorrellStochastic Modeling Seminar35 Mobile Robot Localization http://www.cs.washington.edu/ai/Mobile_Robotics/mcl/2 Observations are sensor data (image, video, sonar, laser rangefinder, etc.) Robot position is unknown. The robot’s position is estimated by correlating sensor data with known maps.

36. Darryl MorrellStochastic Modeling Seminar36 Tracking Articulated Objects http://www.dai.ed.ac.uk/CVonline/LOCAL_COPIES/ RINGER1/mocap_overview.html Observations are video sequences from two cameras. Unknowns are positions and velocities of model components

37. Darryl MorrellStochastic Modeling Seminar37 Tracking with Networks of Distributed Sensors http://www.parc.xerox.com/spl/projects/cosense/ Targets are tracked using an ad hoc network of distributed micro-sensors.

38. Darryl MorrellStochastic Modeling Seminar38 Other Applications Channel equalization Estimation of parameters of multiple chirp signals Multiple target tracking Bearing’s-only target tracking Track before detect target tracking Image segmentation

39. Darryl MorrellStochastic Modeling Seminar39 Presentation Outline Applications of particle filters Fundamental concepts Anatomy of a simple particle filter Variations on the simple particle filter Pros and Cons of particle filters Application to configuration of a foveal sensor Conclusions

40. Darryl MorrellStochastic Modeling Seminar40 Fundamental Concepts Bayesian inference Monte Carlo samples Importance Sampling Resampling

41. Darryl MorrellStochastic Modeling Seminar41 Bayesian Inference X is unknown-a random variable or set (vector) of random variables Z is observed-also a set of random variables We wish to infer X by observing Z. The probability distribution p(x) models our prior knowledge of X. The conditional probability distribution p(z|x) models the relationship between Z and X.

42. Darryl MorrellStochastic Modeling Seminar42 Bayes Theorem The conditional distribution p(x|z) represents posterior information about X given Z.

43. Darryl MorrellStochastic Modeling Seminar43 Monte Carlo Samples (Particles) The posterior distribution p(x|z) may be difficult or impossible to compute in closed form. An alternative is to represent p(x|z) using Monte Carlo samples (particles): –Each particle has a value and a weight x x

44. Darryl MorrellStochastic Modeling Seminar44 Importance Sampling Ideally, the particles would represent samples drawn from the distribution p(x|z). –In practice, we usually cannot get p(x|z) in closed form; in any case, it would usually be difficult to draw samples from p(x|z). We use importance sampling: –Particles are drawn from an importance distribution. –Particles are weighted by importance weights.

45. Darryl MorrellStochastic Modeling Seminar45 Resampling In inference problems, most weights tend to zero except a few (from particles that closely match observations), which become large. We resample to concentrate particles in regions where p(x|z) is larger. x x

46. Darryl MorrellStochastic Modeling Seminar46 Presentation Outline Applications of particle filters Fundamental concepts Anatomy of a simple particle filter Variations on the simple particle filter Pros and Cons of particle filters Application to configuration of a foveal sensor Conclusions

47. Darryl MorrellStochastic Modeling Seminar47 Anatomy of a Simple Particle Filter A simple particle filter requires the following: A system state evolution model An observation model Particle computation processes: –Propagate forward in time –Compute weights given observations –Resampling

48. Darryl MorrellStochastic Modeling Seminar48 System State The state represents the unknown whose value we want to infer. For example, –Position (and velocity) of a robot, car, plane,... –Position of articulated model components. The system state at (discrete) time k is denoted x k. The state evolves according to the following dynamics equation: x k+1 = f k (x k, w k )

49. Darryl MorrellStochastic Modeling Seminar49 Observation Model The observation z k may be an image, a frame of video, a radar or sonar measurement, etc. The relationship between the observation and the state is given by the conditional probability distribution p(z k | x k ). This distribution may be derived from a functional relationship between z k and x k : z k = h k (x k, v k )

50. Darryl MorrellStochastic Modeling Seminar50 Objective-Find p(x k |z k,…,z 1 ) The objective of the particle filter is to compute the conditional distribution p(x k |z k,…,z 1 ) To do this analytically, we would use the Chapman-Kolmogorov equation and Bayes Theorem along with Markov model assumptions. The particle filter gives us an approximate computational technique.

51. Darryl MorrellStochastic Modeling Seminar51 Particle Filter Algorithm Create particles as samples from the initial state distribution p(x 0 ). For k going from 1 to K –Sample each particle from a proposal distribution. –Compute weights for each particle using the observation value. –(Optionally) resample particles.

52. Darryl MorrellStochastic Modeling Seminar52 Initial State Distribution x0x0 x0x0

53. Darryl MorrellStochastic Modeling Seminar53 State Update x0x0 x 1 = f 0 (x 0, w 0 ) x1x1 This is one way to sample from a proposal distribution.

54. Darryl MorrellStochastic Modeling Seminar54 Compute Weights x1x1 x1x1 p(z 1 |x 1 ) x1x1 Before After

55. Darryl MorrellStochastic Modeling Seminar55 Resample x1x1 x1x1

56. Darryl MorrellStochastic Modeling Seminar56 Particle Filter Demonstration A target moves from left to right. Two sensors: –Each measures the distance from itself to the target. –Sensors at (30,0) and (0,50) 4000 Particles were used to track the target. The animation on the following slide shows the particles, the true target position, and the estimated target position.

57. Darryl MorrellStochastic Modeling Seminar57 Particle Filter Demonstration

58. Darryl MorrellStochastic Modeling Seminar58 Presentation Outline Applications of particle filters Fundamental concepts Anatomy of a simple particle filter Variations on the simple particle filter Pros and Cons of particle filters Application to configuration of a foveal sensor Conclusions

59. Darryl MorrellStochastic Modeling Seminar59 Variations on this Simple Implementation Use a different importance distribution: –In this implementation, the importance distribution is the predicted state distribution p(x k+1 |z k,…,z 1 ). –Several papers have pointed out that this distribution may not be the best one can use. –If the observation at time k+1 is available, significant improvement in performance can be obtained.

60. Darryl MorrellStochastic Modeling Seminar60 Variations Use a different resampling technique: –Resampling adds variance to the estimate; several resampling techniques are available that minimize this added variance. –Our simple resampling leaves several particles with the same value; methods for spreading them are available.

61. Darryl MorrellStochastic Modeling Seminar61 Variations Reduce the resampling frequency: –Our implementation resamples after every observation, which may add unneeded variance to the estimate. –Alternatively, one can resample only when the particle weights warrant it. This can be determined by the effective sample size.

62. Darryl MorrellStochastic Modeling Seminar62 Variations Rao-Blackwellization: –Some components of the model may have linear dynamics and can be well estimated using a conventional Kalman filter. –The Kalman filter is combined with a particle filter to reduce the number of particles needed to obtain a given level of performance.

63. Darryl MorrellStochastic Modeling Seminar63 Presentation Outline Applications of particle filters Fundamental concepts Anatomy of a simple particle filter Variations on the simple particle filter Pros and Cons of particle filters Application to configuration of a foveal sensor Conclusions

64. Darryl MorrellStochastic Modeling Seminar64 Advantages of Particle Filters Under general conditions, the particle filter estimate becomes asymptotically optimal as the number of particles goes to infinity. Non-linear, non-Gaussian state update and observation equations can be used. Multi-modal distributions are not a problem. Particle filter solutions to inference problems are often easy to formulate.

65. Darryl MorrellStochastic Modeling Seminar65 Disadvantages of Particle Filters Naïve formulations of problems usually result in significant computation times. It is hard to tell if you have enough particles. The best importance distribution and/or resampling methods may be very problem specific.

66. Darryl MorrellStochastic Modeling Seminar66 Presentation Outline Applications of particle filters Fundamental concepts Anatomy of a simple particle filter Variations on the simple particle filter Pros and Cons of particle filters Application to configuration of a foveal sensor Conclusions

67. Darryl MorrellStochastic Modeling Seminar67 A Foveal Sensor Low Acuity Region Foveal Region A foveal sensor has a high acuity area (similar to the fovea of the eye) that can be steered towards a desired location. Target Position

68. Darryl MorrellStochastic Modeling Seminar68 Mathematical Model The foveal sensor is modeled mathematically as: z k = tan -1 (C k (x k -d k )) x k is the target position. d k controls the location of the center of the foveal region. C k controls the width of the foveal region. xkxk zkzk dkdk Foveal Region

69. Darryl MorrellStochastic Modeling Seminar69 Before 1st Observation Position True Position Estimated Initial Position

70. Darryl MorrellStochastic Modeling Seminar70 Observed Collecting 1st Observation Position True Predicted 1Foveal sensor is configured using predicted values. 2Observation is obtained 3Position (and velocity) estimates are computed Estimated

71. Darryl MorrellStochastic Modeling Seminar71 Observed Collecting 2nd Observation Position True Predicted 1Foveal sensor is configured using predicted values. 2Observation is obtained 3Position (and velocity) estimates are computed Estimated

72. Darryl MorrellStochastic Modeling Seminar72 Observed Collecting 3rd Observation Position True Predicted 1Foveal sensor is configured using predicted values. 2Observation is obtained 3Position (and velocity) estimates are computed Estimated

73. Darryl MorrellStochastic Modeling Seminar73 Implementation We implemented a particle filter to estimate the target position from observations. –The foveal region is centered on the predicted target position. –The gain is either set to a constant value or adjusted to include a certain percentage of the particles in the foveal region. The implementation took a few hours. Tuning the filter has taken a few weeks.

74. Darryl MorrellStochastic Modeling Seminar74 Comparison with Previous Foveal Sensor A two dimensional linear system dynamics model is used. The system state transition matrix is stable. The following plot shows curves of constant estimation error as a function of process and observation noise variance: –Stat fixed gain-Kalman filter implementation of fixed gain sensor –PF fixed gain-Particle filter implementation of a fixed gain sensor –PF Var. gain-Particle filter implementation of an adaptive gain sensor

75. Darryl MorrellStochastic Modeling Seminar75 Curves of Constant Error

76. Darryl MorrellStochastic Modeling Seminar76 Discussion of Results Adaptive acuity gives better performance than fixed acuity. The particle filter implementations do not perform well with very small observation noise variances. –The number of particles is too small for very sharply peaked observation densities-few particles fall within the peaks. –Several approaches to improve the performance for small observation variances are currently under investigation.

77. Darryl MorrellStochastic Modeling Seminar77 Fixed vs. Adaptive Acuity The foveal sensor collects observations of position. The acuity of the foveal region is adjusted so that 80% of the predicted particle positions fall into the foveal region. The gain of the foveal region is smoothed using a low-pass filter with an exponentially decaying impulse response. We show plots of the average squared estimate error as a function of time for –Fixed acuity foveal sensor for gains of 0.25, 1, and 4 –Adaptive acuity foveal sensor

78. Darryl MorrellStochastic Modeling Seminar78 Average Estimate Error

79. Darryl MorrellStochastic Modeling Seminar79 Presentation Outline Applications of particle filters Fundamental concepts Anatomy of a simple particle filter Variations on the simple particle filter Pros and Cons of particle filters Application to configuration of a foveal sensor Conclusions

80. Darryl MorrellStochastic Modeling Seminar80 Conclusions Particle filters (and other Monte Carlo methods) are a powerful tool to solve difficult inference problems. –Formulating a filter is now a tractable exercise for many previously difficult or impossible problems. –Implementing a filter effectively may require significant creativity and expertise to keep the computational requirements tractable.