1. Presented by: Cindy Yan EE6358 Computer Vision
Dynamic Vision
Presented by: Cindy Yan
EE6358 Computer Vision

2. Out Line: Change detection Segmentation using motion
Difference Picture (DP)
Accumulative Difference Picture (ADP)
Segmentation using motion
Match correspondence
Point matching
Line matching
Optical Flow in Motion analysis
Tracking
Computer Vision

3. Dynamic Scene Analysis System
Input: a sequence of image frames
Each frame represents an image of the scene at a particular instants of time
Changes in a Scene
Camera motion
Object motion
Computer Vision

4. Camera and world setup
SCSO
SCMO
MCSO
MCMO
Computer Vision

5. 1. Change Detection
Detection of changes in two successive frames of a sequence
Changes can be detected at different levels: pixel, edge, regions.
Computer Vision

6. Difference Pictures:
Compare the corresponding pixels of the two frames
A binary difference picture d(i,j) between f1(i,j) and f2(i,j) is obtained by:
Computer Vision

7. Computer Vision

8. Size Filter Difference picture results in too many noise pixels.
Pixels that belong to a connected component and larger than a minimum size are retained for further analysis.
Size filter will reduce the noise, but also filters some desired signals such as slow or small moving objects.
Computer Vision

9. Accumulative Difference Pictures (ADP)
Analyzing the changes over a sequence of frames.
Comparing every frame of an image sequence to a reference frame.
Increase the entry in the accumulative difference picture by 1, whenever the difference exceeds a threshold
ADP0 = 0
ADPk(x,y)= ADPk-1(x,y)+DP1k(x,y)
Computer Vision

10. Computer Vision

11. 2. Segmentation Using Motion
For a stationary camera:
Segmentation involves separating moving components in the scene from stationary components
Segmentation may be performed using region-based or edge-based approaches
Computer Vision

12. Time-Varying Edge Detection
Moving edge: an edge that moves
Moving edges can be detected by combining the temporal and spatial gradients using a logical AND operator t X Y
Computer Vision

13. Performance of the edge detector
Slow moving edges will be detected if they have good contrast
Poor-contrast edges will be detected if they move well
|D|
|E| T
Computer Vision

14. 3. Match Correspondence
Find corresponding image features (points or lines) from two image frames that correspond to the same features in 3D scene.
Relaxation Labeling
A constraint propagation approach to solve the correspondence problem
Proper labels must be assigned to the object in the image
Computer Vision

15. Two concerns in matching problem
How are points selected for matching?
What are the features that are matched?
How are the correct match chosen?
What constraints are placed on the displacement vectors?
Computer Vision

16. Relaxation Labeling Process
Proper labels must be assigned to the objects in the image
Define R,C,L,P for each node.
R: contains all possible relations among the nodes
C: represents the compatibility among these relations
L: contains all labels that can be assigned to nodes
P: represents the set of possible levels that can be assigned to a node at any instant in computation
Decide which of the possible interpretations is correct on the basis of local evidence
Computer Vision

17. Disparity Computations
Matching Problem: pair a point pi=(xi,yi) in the first image with a point pj=(xj,yj) in the second image.
Disparity between them is the displacement vector between the two points.
Dij = (xi-xj,yi-yj)
Computer Vision

18. How are points selected for matching?
Discreteness:
A measure of the distinctiveness of individual points
Similarity:
A measure of how closely two points resemble one another
Consistency:
A measure of how well a match conforms to nearby matches
Computer Vision

19. Discrete feature
Discreteness means that features should be isolated points
The discrete feature points can be selected using any corner detector or a feature detector such as Moravec interest operator.
Computer Vision

20. Moravec interest operator
Detects points at which intensity values are varying quickly in at least one direction
Compute sum of the squares of pixel differences in four directions (horizontal, vertical and both diagonals) over a 5 by 5 window
Computer Vision

21. Moravec interest operator (cont.)
Compute the minimum value of these variances
Suppress all values that are not local maxima
Apply a threshold to remove weak feature points
Computer Vision

22. Moravec interest operator (cont.)
After finding all the potential matches, pair each feature point in the first image with all points in the second image within some maximum distance
This will eliminate many connections from the complete bipartite graph.
Computer Vision

23. Point Matching
Defines object set O = {o1,o2,…om} from image points of frame 1, each element is a node. Define label set L={l1,l2,…,ln } from points of frame 2.
Establish relationship set among the nodes of object nodes, such as neighboring points.
Establish an initial match set:
M(0)={(<o1,l1>,<o1,l2>,…<o1,ln>), ……
(<oi,l1>,<oi,l2>,…<oi,ln>)
(<om,l1>,<om,l2>,…<om,ln>)};
Computer Vision

24. Point matching (cont.)
The set of potential matches form a bipartite graph.
The goal of the correspondence problem is to remove all other connections except one for each node.
Computer Vision

25. Consistency measurement
Based on:
geometric relation among node in image.
gray level or gradient in the original image of the node.
Compute similarity (or disparity) of each node with respect to matched pair. E.g.,
Probability of match between oi and li is:
Computer Vision

26. Consistency measurement (cont.)
Update match set M(k) iteratively:
If the similarity of <oi,li> is high, encourage the match of its consistent nodes. Otherwise discourage the match of its consistent nodes.
Remove the match pair of small similarity (small match probability pi(k)(l|i)from match set M(k)
Repeat above steps until each node has no more than one label in M(k)
Computer Vision

27. Line Matching
Given: Two set of Lines in image A and image B respectively.
Find: Unique correspondences of lines between image A and B.
Computer Vision

28. Matching function Position disparity. Relative position in an image:
where is edge direction.
Position disparity between two sub-sets of image lines from images A and B.
Computer Vision

29. Line Matching (cont.)
Orientation disparity:
Computer Vision

30. Line Matching (cont.) Other disparity: Length of Line
Intensity of original image
  Contrast
  Steepness
  Straightness (residues of Least squares)
Computer Vision

31. Kernel Match:
Match a small sub-sets from image Lines of frames A and B, for robustness consideration of the kernel,
Number of lines should be no less than 3.
Lines should be long (stable).
Lines should not be paralleled.
Lines should be separated as much as possible.
Minimize the match function over selected subjects between two image frames.
x xth attribute, such as position ,orientation.
α weight
Computer Vision

32. Match Expansion:
Once kernel matching is completed, the line correspondences obtained will serve a reference for the match of remaining lines.
Choose a longest line from unmatched line of image A. Add it into the subset of matched kernel of image A, calculate match functions for every unmatched line in image B. The line of image B with minimum match function is considered a matched line. Add this matched pair of lines into matching kernel and repeat the process until no further line needs to match.
Computer Vision

33. 4. Optical Flow in Motion analysis
Optical Flow reflects the image changes due to motion during a time interval dt.
Computer Vision

34. Basic elements of motion
a). Translation at constant distance from observer
b). Translation in depth relative to observer
c). Rotation at constant distance about the view axis
d). Rotation of a planar object perpendicular to the view axis
Computer Vision

35. FOE: focus of expansion
If several independently moving objects are present in the image, each motion has its own FOE
Computer Vision

36. Mutual Velocity of observer
Cx=u, cy=v, cz =w are mutual velocities in directions x,y,z respectively.
x’, y’ be the image co-ordinates
x0,y0,z0 be position of some point at time t0
The position of the same point at t is:
Computer Vision

37. FOE determination:
Assume motion directed toward an observer; as t  -∞
The motion can be traced back to the originating point at infinite distance from the observer.
The motion toward an observer continues along straight lines and the originating point in the image plane is:
Computer Vision

38. Depth Determination
Assume points of the same rigid object and translational motion. At least one actual distance value is known.
Assume an object moving towards the observer
Computer Vision

39. Finding the real word co-ordinate
Assume motion is along the camera optical axis.
Computer Vision

40. 6. Object Tracking
Refers to tracking of object motion in a sequence of frames
Given: m objects moving in scene, a sequence of n image frames is taken from the scene.
Find: the trajectories of each object in the image sequence.
Computer Vision

41. Path coherence assumption:
Assume:
Change of object location is small
Change of scalar velocity is small
Change of moving direction is small
Principles of Path Coherence function Ф represents a measure of agreement between the derived object trajectory and the motion constraints
Function value always positive
reflects local absolute angular deviations of the trajectory
respond equally to positive and negative velocity changes
Normalized
Computer Vision

42. Path Coherence function Ф
Trajectory Ti of an object i:
In Vector form:
Deviation function:
The deviation Di of the entire trajectory of the object i is:
Computer Vision

43. Path coherent function:
For m trajectories of m moving object in the image sequence, the overall trajectory deviation D:
Path coherent function: +
Computer Vision

44. References:
Motion Analysis:
Lecture:
Motion Analysis:
Dynamic Motion:
Computer Vision