1. Motion Detection And Analysis
Michael Knowles
Tuesday 13th January 2004

2. Introduction Brief Discussion on Motion Analysis and its applications
Static Scene Object Tracking
Motion Compensation for Moving-Camera Sequences

3. Applications of Motion Tracking
Control Applications
Object Avoidance
Automatic Guidance
Head Tracking for Video Conferencing
Surveillance/Monitoring Applications
Security Cameras
Traffic Monitoring
People Counting

4. Two Approaches Optical Flow Object-based Tracking
Compute motion within region or the frame as a whole
Object-based Tracking
Detect objects within a scene
Track object across a number of frames

5. My Work Started by tracking moving objects in a static scene
Develop a statistical model of the background
Mark all regions that do not conform to the model as moving object

6. My Work
Now working on object detection and classification from a moving camera
Current focus is motion compensated background filtering
Determine motion of background and apply to the model.

7. Static Scene Object Detection and Tracking
Model the background and subtract to obtain object mask
Filter to remove noise
Group adjacent pixels to obtain objects
Track objects between frames to develop trajectories

8. Background Modelling

9. Background Model

10. After Background Filtering…

11. Background Filtering My algorithm based on:
“Learning Patterns of Activity using Real-Time Tracking” C. Stauffer and W.E.L. Grimson. IEEE Trans. On Pattern Analysis and Machine Intelligence. August 2000
The history of each pixel is modelled by a sequence of Gaussian distributions

12. Multi-dimensional Gaussian Distributions
Described mathematically as:
More easily visualised as:
(2-Dimensional)

13. Simplifying….
Calculating the full Gaussian for every pixel in frame is very, very slow
Therefore I use a linear approximation

14. How do we use this to represent a pixel?
Stauffer and Grimson suggest using a static number of Gaussians for each pixel
This was found to be inefficient – so the number of Gaussians used to represent each pixel is variable

15. Weights Each Gaussian carries a weight value
This weight is a measure of how well the Gaussian represents the history of the pixel
If a pixel is found to match a Gaussian then the weight is increased and vice-versa
If the weight drops below a threshold then that Gaussian is eliminated

16. Matching
Each incoming pixel value must be checked against all the Gaussians at that location
If a match is found then the value of that Gaussian is updated
If there is no match then a new Gaussian is created with a low weight

17. Updating
If a Gaussian matches a pixel, then the value of that Gaussian is updated using the current value
The rate of learning is greater in the early stages when the model is being formed

18. Colour Spaces
If RGB is used then the background filtering is sensitive to shadows
The use of a colour space that separates intensity information from chromatic information overcomes this
For this reason the YUV colour space is used

19. Colour Spaces
Background and Frame:
Channel Differences:

20. Isolate Objects
Groups of object pixels must be grouped to form objects
A connected components algorithm is used
The result is a list of objects and their position and size

21. Track objects Objects are tracked from frame to frame using: Location
Direction of motion
Size
Colour

22. The Story So Far… Basic principle of background filtering
Stages necessary in maintaining a background model
How it is applied to tracking

23. Moving Camera Sequences
Basic Idea is the same as before
Detect and track objects moving within a scene
BUT – this time the camera is not stationary, so everything is moving

24. Motion Segmentation
Use a motion estimation algorithm on the whole frame
Iteratively apply the same algorithm to areas that do not conform to this motion to find all motions present
Problem – this is very, very slow

25. Motion Compensated Background Filtering
Basic Principle
Develop and maintain background model as previously
Determine global motion and use this to update the model between frames

26. Advantages Only one motion model has to be found
This is therefore much faster
Estimating motion for small regions can be unreliable
Not as easy as it sounds though…..

27. Motion Models
Trying to determine the exact optical flow at every point in the frame would be ridiculously slow
Therefore we try to fit a parametric model to the motion

28. Affine Motion Model
The affine model describes the vector at each point in the image
Need to find values for the parameters that best fit the motion present

29. Minimisation of Error Function
If we are to find the optimum parameters we need an error function to minimise:
But this is not in a form that is easy to minimise…

30. Gradient-based Formulation
Applying Taylor expansion to the error function:
Much easier to work with

31. Gradient-descent Minimisation
If we know how the error changes with respect to the parameters, we can home in on the minimum error
Various methods built on this principle:

32. Applying Gradient Descent
We need:
Using the chain rule:

33. Robust Estimation
What about points that do not belong to the motion we are estimating?
These will pull the solution away from the true one

34. Robust Estimators
Robust estimators decrease the effect of outliers on estimation

35. Error w.r.t. parameters
The complete function is:

36. Aside – Influence Function
It can be seen that the first derivative of the robust estimator is used in the minimisation:

37. Pyramid Approach
Trying to estimate the parameters form scratch at full scale can be wasteful
Therefore a ‘pyramid of resolutions’ or ‘Gaussian pyramid’ is used
The principle is to estimate the parameters on a smaller scale and refine until full scale is reached

38. Pyramid of Resolutions
Each level in the pyramid is half the scale of the one below – i.e. a quarter of the area

39. Out pops the solution….
When combined with a suitable gradient based minimisation scheme…

40. Problems with this approach
Resampling the background model:
Model cannot be too complex
Resampling will bring in errors
Motion model is only an estimate of what is really happening
Can lead to false object detection – particularly close to boundaries

41. Background Model Design
The background model needs to be robust to these problems
We need some way to differentiate between genuine object detections and false ones from motion model and background model errors

42. My Approach
Rather than updating model values with current, matched values replace them
In this way resampling errors are not allowed to accumulate

43. Aside – ‘Real Time’
The ability to process a sequence in real-time is dependent on THREE key factors:
The speed of the algorithm
The frame rate required
The number of pixels

44. Recap Introduction to motion analysis
Principles of background modelling
Example of a static scene tracker
Discussion of motion estimation
Shortcomings when applied to background filtering