1. Performance of Optical Flow Barron, Fleet and Beauchemin IJCV 12:1, 1994 http://www.csd.uwo.ca/faculty/barron/

2. Performance of Optical Flow Evaluation of different optical flow techniques – Accuracy, reliability, density of measurements A common set of synthetic and real sequences Several optical flow methods – Differential – Matching – Energy-based – Phase-based

3. Performance of Optical Flow Accurate and dense velocity measurement Accurate 2d motion filed estimation is ill- posed – Inherent differences between the 2D motion field and intensity variations Only qualitative information can be extracted

4. Optical Flow Process Three stages – Perfiltering or smoothing with low-pass/band-pass filters in order to extract signal structure of interest enhance the signal-to-noise ratio – Extraction of basic measurements Spatiotemporal derivatives Local correlation surface – Integration of measurements to produce 2D flow field Often involves assumptions about the smoothness of the underlying flow field

5. Differential Techniques First-order derivatives and based on image translation Intensity is conserved Normal velocity

6. Differential Techniques Second-order differential Stronger restriction than first-order derivatives on permissible motion field Can be combined with 1 st order in isolation or together (over- determined system) Velocity estimation from 2 nd -order methods are often assumed be to sparser and less accurate than estimation from 1 st -order methods

7. Differential Techniques Additional constraints – Fits the measurements in each neighborhood to a local model for 2d velocity Using least squares minimization or Hough transform – Global smoothness

8. Differential Techniques must be differentiable – Temporal smoothing at the sensors is needed to avoid aliasing – Numerical differentiation must be done carefully If aliasing can not be avoided in image acquisition – Apply differential techniques in a coarse-to-fine manner

9. Horn and Schunck Combine gradient constraint with a global smoothness term, minimizing

10. Horn and Schunck Relatively crude form of numerical differentiation can be source of error Spatiotemporal smoothing – Gaussian prefilter with 1.5 pixels in space and 1.5 frames in time 4-point central differences for differentiation – mask

11. Lucas and Kanade Weighted least squares Fixed velocity in a small neighborhood Minimizing

12. Lucas and Kanade When is nonsingular, Weighted least squares estimates of v from estimates of normal velocities Confidence measure

13. Lucas and Kanade Spatiotemporal smoothing – Gaussian prefilter with 1.5 pixels-frames 4-point central differences for differentiation – mask Spatial neighborhood 5x5 pixels Window function W(x) – (0.0625, 0.25, 0.375, 0.25, 0.0625)

14. Lucas and Kanade Identify the unreliable estimates by eigenvalues of – If – If, compute normal velocity v=sn From LS minimization – Otherwise, do not compute velocity

15. Nagel First to use second-order derivatives to measure optical flow – Basic measurements and global smoothness – Oriented smoothness constraint – Attenuates the variation of the flow in the direction perpendicular to the gradient

16. Nagel Gauss-Seidel iterations

17. Nagel Weight matrix

18. Nagel Spatiotemporal smoothing 4-point central differences for differentiation Velocity derivatives – 1 st order: 2 point central difference ½(1,0,-1) – 2 nd order: cascades of 1 st order derivatives Barron’s implementation

19. Uras, Girosi, Verri and Torre Local solution to Solved wherever the Hessian H is nonsingular 8x8 pixel regions – For each region, select 8 estimates that best satisfy – Choose the estimate with the smallest condition number k(H) as the velocity for the entire region

20. Uras et al. Presmooth using Gaussian – 3 pixels in space and 1.5 frames in time Derivatives of I and v – 4 point central difference operators Confidence measurement – They use k(H) – Barron et al. found det(H) is more reliable Barron’s implementation

21. Region-Based Methods Accurate numerical differentiation may be impractical because of noise, a small number of frames, aliasing Region-based approaches – Define v as the shift that yields the best fit between image regions at different times – Best match  maximizing a similarity measure

22. Region-Based Matching Sum-of-squared difference (SSD) Cross-correlation, NCC… Discrete 2D windowInteger values (dx, dy)

23. Anandan Based on Laplacian pyramid – Allows the computation of large displacement between frames – Help enhance image structure (edges.. ) Coarse-to-fine SSD-based matching strategy – Coarsest level: displacement be 1p/f or less – SSD minima in 3x3 search space using 5x5 Gaussian of W(x) – Subpixel displacement are computed by finding the minimum of a quadratic surface parameters

24. Anandan Confidence measures of the SSD surface at the minimum – – S_min: SSD value at the minima – k_1 = 150, k_2=1, k_3 =0 Principle curvatures

25. Anandan Additional smoothness constraint Minimize Gauss-Seidal iterations The direction of min and max curvature on the SSD surface at the minima The displacement from the higher level

26. Anandan Matching and Smoothing are performed at each level of the Laplacian pyramid Confidence measure – Try to use c_min and c_max suggested by Anandan, but not reliable

27. Singh Two-stage matching method – First, SSD with 3 adjacent band-pass filtered image Converts SSD0 into a probability distribution Average out spurious SSD minima due to noise or periodic texture

28. Singh Subpixel velocity: mean of the distribution – Averaged over the integer displacement d Coarse-to-fine strategy Confidence measures: eigenvalues of the inverse covariant matrix

29. Singh Step1, computed SSD for a wide range of integer displacement, N=4 – (4N+1)x(4N+1) SSD surface to (2N+1)x(2N+1) subregions Step2: propagate velocity using neighborhood constraints Barron’s implementation Gauss function of distance, better results with w=2 than w=1

30. Singh Covariance matrix Final velocity – S_c, v_c are derived from intensity data in step1 – S_n, v_n Barron’s implementation Matrix inverse: replace singular values less than 0.1 by 0.1 to avoid singular systems

31. Singh Confidence measures – Eigenvalues of covariance matrix –, serve as confidence measures Rejecting velocities where

32. Energy-Based Methods Based on the output energy of velocity-tuned filters – Also called frequency-based methods owing to the design of velocity- tuned filters in the Fourier domain Fourier transform of a translating 2d pattern is – All non-zero power associated with a translating pattern lies on a plane through the origin in frequency space Equivalent to correlation-based method, gradient-based method of Lucas and Kanade FT of I(x,0) Temporal frequency K=(k_x,k_y) spatial frequency

33. Heeger Least-squares fit of spatiotemporal energy to a plane in frequency space – Extract local energy using Gabor-energy filters, with 12 filters at each of several spatial scales, tuned to different spatial orientations and temporal frequencies Ideally, for a single translational motion, the response of these filters are concentrated about a plane in frequency space

34. Heeger Expected response of a Gabor-energy filter tuned to frequency for translating white noise as a function of velocity The standard deviations of Gaussian component of Gabor filter

35. Heeger The set of filters with the same orientation tuning: Sum of measured and predicted energies from filter j in the set of M_i ： Least-squares estimate for (u,v): minimize

36. Heeger Two ways of minimizing – Non-linear minimization using Newton’s method: unsatisfactory results – Rarely get convergence if the measurement error was much over 10% Modified minimizing – Construct distribution for a range – The minima of the distribution gives the subpixel velocity estimate – Ad hoc method involves multi-resolution minima selection is used to compute subpixel minima

37. Phase-Based Techniques Velocity is defined in terms of the phase behaviour of band-pass filter outputs First developed by Fleet and Jepson

38. Waxman and Wu and Bergholm Apply spatiotemporal filters to binary edges maps to track edges in real-time Convected activation profile A(x,t) Track level contours of A using differential methods – Spatial gradient of A = 0 at edge locations – 2 nd order approaches to estimate Edge map

39. Waxman and Wu and Bergholm Implementation – Central Gaussian of the DOG had a standard deviation of 1.5 pixels-frames – Activation profile Require 7 frames – Waxman et al, multiple method to choose the best velocity at an edge location. For various values (1.0, 1.5, 2.0), choose the velocity that maximizes

40. Waxman and Wu and Bergholm Confidence measure: Hessian of A (Gaussian curvature of A ) If, compute full velocity Otherwise, compute normal velocity

41. Fleet and Jepson Define component velocity in terms of the instantaneous motion normal to level phase contours in the output of band-pass velocity-tuned filters Band-pass filters: to decompose the input signal according to scale, speed and orientation

42. Fleet and Jepson 2D velocity Phase derivatives

43. Fleet and Jepson Motivation – The phase component of band-pass filters outputs is more stable than the amplitude component when small deviations from image translations Unstable phase – Instabilities occur in the neighborhoods about phase singularities – Detect with constraint on the instantaneous frequency of the filter output and its amplitude variation in space-time – Also a signal-to-noise constaints

44. Fleet and Jepson Given component velocity estimates from different filter channels, a linear velocity model is fit for each local region – Collect reliable velocity estimates from 5x5 neighborhoods, – Estimate the linear velocity model in a LS sense Additional constraint to ensure sufficient local information – Conditioning of linear system < 10 – Residual LS error < 0.5

45. Experimental Technique Test sequences – Real sequences – Synthetic sequence – With 2D motion field known Error metric – Angular measures of error

46. Synthetic Image Sequence 2D motion fields and sequence properties can be controlled and tested in a methodical fashion – Clean signals No occlusion, specularity, shadowing, transparency, etc Optimistic bound on the expected errors with real image sequence

47. Sinusoidal Inputs Superposition of two sinusoidal plane-waves Results – Spatial wavelength of 6 pixels, with – Orientations of 54°and -27°, – Speeds of 1.63 and 1.02 pixel/frame Two sinusoidal inputs – Translates with velocity – Another plaid pattern with wavelength of 16 pixels/cycle and velocity Sinusoid 1 Results

48. Translating Sqaures Translating squares (width of 40 pixels) Velocity – Uniform velocity – Sometimes Helps illustrate the aperture problem and the inherent spatial smoothing in the difference techniques Square 2 Results

49. Sinusoidal and Squares Sinusoidal inputs – Dense in space – Sparse in frequency space Squares – Concentrated in space along the edges – Richer in frequency spectra Sinusoid 1 Square 2 Barron et al.

50. 3D Camera Motion and Planar Surface Textured planar surface Simulated translational camera motion Translating tree Diverging Tree

51. 3D Camera Motion and Planar Surface (a) Surface texture (b) Translating tree (c) Diverging tree Camera movenormal to line of sight along X-axisalong its line of sight Velocity directionall parallel with image x-axisFocus of expansion is at the image center velocity1.73~2.26 pixel/frame1.29 p/f on the left to 1.86 p/f on the right David Fleet

52. Yosemite Sequence Motion – Divergent motion in the upper-right – Clouds translates to the right with 1 p/f – Velocities in the lower-left ~ 4 p/f Difficult sequence – Velocities in a large ranges – Occluding edges between mountains and at the horizon Lynn Quam

53. Real Image Sequences SRI trees NASA sequence Rotating Rubik Cube Hamburg Taxi

54. SRI Trees Challenging because – Poor resolution – Amount of occlusion – Low contrast – Velocities ~ 2 pixel/frame

55. NASA Sequence Primarily dilational Velocities < 1 pixel/frame

56. Rotating Rubik Cube The cube is rotating counterclockwise on a turntable Velocities on the table 1.2~1.4 p/f Velocities on the cube 0.2~0.5 p/f

57. Hamburg Taxi Sequence Four moving objects Speeds – 1.0 p/f – 3.0 p/f – 0.3 p/f http://i21www.ira.uka.de/image_sequences/

58. Error Measurement Angular measure of error Angular error Correct velocityEstimate VelocityDisplacement per time unit VelocitySpace-time direction vector in units of (pixel, pixel, frame)

59. Error Measurement Angular measure of error Advantage – It handles large and very small speeds without the amplification inherent in a relative measure of vector differences Disadvantages – Have bias: directional errors at small speeds do not give as large an angular error as similar directional errors at higher speeds Angular error

60. Error Measurement

61. Complementary measure of normal velocity – Linear relationship between normal velocity and 2-d velocity – All component velocities generated by a translating texture pattern should ideally lie on the plane normal to – Angle between measured component velocity and the constraint plane

62. Error Measurement Many ways in which error behavior may be reported – For synthetic sequence Extract subsets of estimates using confidence measures and then report the densities of them along with their mean error and standard deviations – For real image sequence Show computed flow field and discuss qualitative properties

63. Experimental Results Synthetic image sequences, known velocity field Error statistics between estimates and ground truth – Mean ( ) and standard deviation ( ) Density of measurements for subsets of the estimates extracted using confidence measures as threshold

64. Sinusoid I

65. Generally very good Relative dense, homogeneous structure of the input – Most flow estimates are not thresholded by confidence measure No smoothness

66. Sinusoid I Modified method with improved numerical differentiation, performed better Accuracy of original H-S method approaches the modified method as the spatial wavelength is increased (Sinusoid 2, 0.97 °± 2.62 °) Large standard deviations are not very significant as they are caused by directional errors near the image boundary Performance related on ƛ, when ƛ=100, results were noticeably worse. Here ƛ=0.25

67. Sinusoid I Similar accuracy to that produced by modified Horn and Schunck algorithm, which shares the same numerical differentiation

68. Sinusoid I The results are also good Get more accurate results when Sinusoidal 2 were used as better derivative estimation is possible ( 0.04 ° ± 0.23 °) Results were sensitive to parameters: results were significantly worse with larger values of a larger values of a

69. Sinusoid I Differential techniques works well on sinusoidal inputs, the matching techniques did not accurate direction, but poor speed estimates Main problem ---- aliasing in the construction of Laplacian pyramid: although complete, the Laplacian pyramid produces band-pass channels (levels) that contain substantial aliasing when considered independently of one another

70. Sinusoid I Only when different levels are combined Aliasing cancel to provide accurate reconstruction With sinusoidal inputs and a coarse-to-fine control strategy on the Laplacian pyramid Aliasing causes major errors at coarse levels that are then propagated systematically to finer levels

71. Sinusoid I Same problems for Singh if implemented with a Laplacian pyramid Multiple local minima in the SSD surface with nearly periodic inputs. The SSD surface is initially evaluated at a small number of integer displacements  the global minima may fall midway between integer displacement,  other minima may be mistaken for global minima if they occur closer to a integer displacement The sampling problem occurs less frequently in natural images which lack the exact periodicity, but sampling problem will continue to occur unless finer sampling and interpolation are used

72. Sinusoid I Heeger’s technique – Reasonable results can be expected when input frequencies matches those in the pass-band to which the filters are tuned – Required Assumption: the input has a flat amplitude spectrum (violated by the sinusoid inputs here) Violation is most evident when the frequencies of the component sinusoids are not close to the filter tunings Sinusoid I: no results Good for others: sinusoid with orientations of 0°and 90°, speeds of 1 p/f, spatiotemporal wavelength of 4 pixels/cycle, errors ( 3.24 °± 0.05 °) with density of 24.3%

73. Sinusoid I To obtain good results with this zero-crossing algorithm, one must choose the standard deviation of the activation kernel so thatzero-crossing algorithm it is small enough to prevent interaction between adjacent edges and yet big enough to track each edge over time Zero-crossing must be localized to sub-pixel accuracy (not done by Waxman et al.) in order to obtain good qualitative results when the underlying motion is not integer multiple of pixels Sinusoid 2 satisfy this, errors ( 0.04 °± 0.03 °) with a density of 11.94%, reflecting the density of edge location

74. Sinusoid I Spatiotemporal wavelength of the sinusoid closely matches those to which their filters are tuned. The results are very goodtuned With general inputs, when input signals have local power concentrated near the boundary of a filter’s amplitude spectra, slight errors appear, as a bias in the component estimates toward the velocity tuning of the filters

75. Translating Square 2 Expect normal estimates along the edges and 2d velocities only at the corners Square 2

76. Lack of discrimination by the algorithm between measurements of normal velocity v.s. 2d velocity

77. Translating Square 2 Poor results for several methods – Differential methods Do not have a way of segmenting the measurements into 2d flow, normal velocity, or unreliable estimates

78. Translating Square 2 integrates measurements locally with a clear means of segmenting normal from 2d velocities based on the eigenvalues of the normal matrixnormal matrix

79. Translating Square 2 Use confidence measure based on the spatial Hessian of the smoothed image sequence Higher density due to using a single estimate for each 8x8 region but limits the spatial resolution of the flow field

80. Still LACK of discrimination by the algorithm between measurements of normal velocity v.s. 2d velocity, even with the confidence measure

81. Translating Square 2 Visually pleasing but somewhat inaccurate The common aperture problem with matching methods SSD minima found at integer displacement is extremely sensitive to small variations along the edges Even with good confidence at step 1, the poor estimate will corrupt in step 2

82. Translating Squares Square 1 with integer speeds, Square 2 has subpixel motion Most techniques have similar performance on them – Waxman et al.: poorly on Square 2 because of the implementation lacks of subpixel resolution

83. Square 2 Normal Velocity

84. 2d Estimates from level 1 are more accurate than level 0 Correct velocity coincides with the appropriate velocity range for level 1

85. Translating Squares Provide a clear way of examining the normal velocity estimate as distinct from the 2d velocity estimate Provide a clear way of examining the normal velocity estimate as distinct from the 2d velocity estimate Lucas and Kanade, provide two sources of normal estimates explicitly Lucas and Kanade – Gradient constraint – LS minimization

86. Square 2 Normal Velocity Density as two quantities 17.6%, 65.4%: the density of positions where one or more normal velocities is recovered 1.1, 4.2: the average number of velocities at a single point

87. Realistic Synthetic Data General behaviour of the techniques is similar with above synthetic sequence Modified Horn and Schunck with presmoothing and improved numerical differentiation – Large smoothness parameter yielded somewhat poorer results – Still less accurate than Lucas-KanadeLucas-Kanade Differs in the method used to combine normal constraints Confidence measure based on eigenvalues of the normal equations A’W^2A performs well

88. Translating tree

89. Diverging Tree

90. Quantization Error Gradient-based algorithms – Initial implementation Quantize the Gaussian smoothed sequence with 8- bit/pixel, prior to gradient computation and LS minimization  noisy derivatives Velocity errors – Grew 40%~50% for Lucas-Kanade – Larger for Horn and Schunck (more sensitive to noise)

91. Translating Tree Horn and Schunck’s method of combining normal constraints( the global smoothness constraint) is significantly more sensitive to noise than the local least squares method by Lucas and Kanade 2 nd order technique – Good results on translating tree (both accurate and dense) – Poor on diverging tree, and Yosemitediverging treeYosemite 1 st order constraint is valid for smooth deformations of the input 2 nd order constraints are based on the conservation of the intensity gradient, invalid for rotation, dilation and shear Aliasing of Yosemite sequence makes accurate 2 nd order differentiation difficult

92. Nagel Produce good results Confidence measure is not entirely successful Large threshold  more accurate but less dense Diverging tree: 1.0 threshold -> poorer results – 2 nd order derivatives of intensity and velocity are small for most cases, --> similar results to Horn and Schuck’s

93. Matching algorithms Both methods produce good results on translating tree (Singh’s > Anandan’s)translating tree – Larger neighborhood support for Singh’s algorithm If use 3x3 regions instead of 5x5 regions, errors increase to 2.13 °± 5.15 ° (stage 1) and 1.35 °± 1.68 °(stage 2) Confidence measures – Anandan’s based on c max, and c min is not reliable Anandan’s – Singh’s: inverse eigenvalues of covariance matrix at stage 1 is useful, but inverse eigenvalues of covariance matrix is inefficient Singh’s: Small changes in a threshold based on the largest eigenvalue dramatically change the density of the estimates

94. Matching Algorithms Poorer results on Diverging TreeDiverging Tree – Singh’s: about an order of magnitude worse, especially at step 1 Some due to aliasing and confusion between normal and 2D velocities Most due to subpixel inaccuracy: errors at noninteger displacements are often two or three time larger than those at integer displacement – Diverging tree: a wide range of velocities – Translating tree: close to integer displacement Use coarser temporal sampling Coarse-fine approach

95. Heeger Results from different levels – level 1 of the pyramid for translating treetranslating tree Input speeds coincide with its velocity range of 1.25~2.5 p/f – Level 0 for Diverging TreeDiverging Tree Most of its velocities were below 1.25 p/f – All three levels for YosemiteYosemite Choose the velocity estimates from the level whose speed range was consistent with the true motion field

96. Diverging Tree Normal/Component velocity results

97. Yosemite 2d velocity

98. Synthetic Data Phase-based method (Fleet and Jepson) produced the most consistently accurate results – Perform extremely well on translating tree and Diverging Treetranslating tree Diverging Tree – Not significantly better on YosemiteYosemite Only 15 frames available, have to increase the tuning frequency of filters to reduce the width of support and increase – Narrow bandwidths  greater sensitivity to aliasing and corruption at high frequencies a compared with the Gaussians used by differential techniques – A significant amount of aliasing in certain regions of the image

99. Yosemite Fleet and Jepson – As the phase stability threshold increases, the 2d velocity errors initially increases, but then decreases significantly Increasing number of component velocities available for 2d velocity computations, Increasing robustness of the minimization slightly Considerable improvement with a tighter constraint on the condition number in the LS system

100. Yosemite Most techniques perform relatively poorly – Aliasing – Occluding boundaries, especially for the horizon If the sky is excluded for analysis, better performance But the density does not change

101. Confidence Measures The importance of confidence measures – All techniques produce velocity estimates with a large range of accuracy Use confidence measures as thresholds to extract subset of velocities that are reliable – Perform well – Useful to distinguish locations at which 2D velocity v.s. normal velocity is measured Justify confidence measures – Error behaviour – Density of estimates

102. First-order Differential Methods Weighted minimization used to estimate 2d v from normal constraints involves an implicit weighting of each normal constraint by the magnitude of its spatial gradient Smaller eigenvalue alone is better measure of confidence – The occurrence of the aperture problem is signalled primarily in the smallest eigenvalue – The sum of two eigenvalues can be arbitrarily large while the system remains singular due to the aperture problem – Although significant errors in gradient measurement are manifested in smaller eigenvalues, there are other sources of error are not, such as difference between the 2d motion field and the velocity of level intensity contours

103. 2 nd -order Differential Methods Uras et al. k(H) while Barron et al. det(H) Uras et al. – Det(H) is more consistently reliable, producing better results on the three realistic synthetic sequence and four natural sequence Anandan cmax and cmin

104. Real Image Data With natural image sequence, it is hard to see the difference between different techniques – Errors of 10% or 20% is hard to discern at this resolution – Other errors, like normal velocities mistaken for 2d velocities Main problem

105. Main Problem For integrate normal constraints with global smoothness constraints – Is the lack of a confidence measure that allows one to distinguish a normal velocity estimate from 2d velocity estimate – Comparing Horn and Schunck with local explict method

106. SRI TreeNASA Rubik cube Hamburg Taxi Horn and Schunck

107. SRI TreeNASA Rubik cube Hamburg Taxi Lucas and Kanade

108. Real Image Data Differential and phase-based algorithm works well – Lucas-Kanade, Uras et al., Fleet and Jepson – Uras et al. Uras et al. Sparser set of estimates, but the density competitive – Fleet and JepsonFleet and Jepson Extremely good at the ground plane toward the front of the SRI tree sequence compared with the above trees

109. SRI TreeNASA Rubik cube Hamburg Taxi Nagel Gaussian filter: 3 in space and 1.5 in time

110. SRI TreeNASA Rubik cube Hamburg Taxi Uras et al.

111. SRI TreeNASA Rubik cube Hamburg Taxi AnandanAnandan no thresholding

112. SRI TreeNASA Rubik cube Hamburg Taxi Singh No thresholding

113. SRI TreeNASA Rubik cube Hamburg Taxi Heeger

114. Based on 3 levels of Gaussian pyramid Choose the estimates with speeds that are consistent from their respective levels of the pyramid If consistent estimates are at more than one levels, chose the lowest level

115. SRI TreeNASA Rubik cube Hamburg Taxi Waxman et al. Gaussian with 1.5 space time

116. SRI TreeNASA Rubik cube Hamburg Taxi Fleet and Jepson

117. Summary Compare the performance of a number of optical flow techniques: density and accuracy 9 algorithms – Differential methods – Region-based matching – Energy-based, – Phase-based Comparison between – Different types of algorithms – Different method of the same concept

118. Summary Both real and synthetic image sequence – Not severely corrupted by spatial and temporal aliasing Comparison – Most reliable: 1 st order, local differential method of Lucas and Kanade Local phase-based method -- Fleet and Jepson – 2 nd order differential method of Uras et al. also performs well – Perform consistently well over all the image sequence With confidence measures at different stages Limitation: lack of reliable confidence measures

119. Differential Approaches Importance of numerical differentiation and spatiotemporal smoothing – Some degree of spatiotemporal presmoothing to remove small amount of temporal aliasing and improve the subsequent derivative estimates Had a marked effect on the quantitative accuracy – Temporal smoothing is particularly useful

120. Differential Approaches Methods that combine local differential constraint to obtain 2d velocity estimates – Local explicit methods (local fit to constant or linear models of v) Superior in both accuracy and computational efficiency More robust with respect to errors in gradient measurement caused by quantization noise (modified Horn and Schunck -- Lucas, kanade) Because of the existence of confidence measure to distinguish estimates of normal velocity and 2d velocity

121. 2 nd order differential methods Produce accurate and relatively dense measurement of 2d velocity Det(H) is a good confidence measure, more effective than its condition number k(H) Inconsistent – Good at predominately translational sequence – Degrades fast as the mount of higher-order geometric deformation in the input increases (compare translating tree and diverging tree )compare translating tree and diverging tree

122. Matching Techniques Generally poorer than good differential methods – SSD-based matching: poor ability to estimate subpixel displacement Good for image translation and higher speeds Poor: small velocities with dilational component – Important to use neighborhood smoothness constraint (Singh, Anandan) – Confidence measurement is not effective

123. Energy-based Techniques Not as reliable as others – Nonlinear optimation in Heeger is extremely sensitive to initial conditions and do not produce reliable resultsHeeger Generally, difficult to use

124. Phase-Based Approaches Fleet and Jepson produced the most accurate results overall However – Sensitive to temporal aliasing because of the frequency tuning of the filter – Potential number of confidence measures Phase stability, SNR Better to combine them to a single measure that would facilitate the LS solution to 2d velocities – High computational load A large number of filter

125. Conditions of Tests Temporal aliasing is not a severe problem and the intensity is differentiable Relatively simple image sequences – Without occlusion, specularities, multiple motions.. – Performance measures should be taken as lower bound on the expected accuracy under general conditions – Most implementations use only one-scale of filtering,  multi-scale implementations