1. PDE Methods for Image Restoration
Lecture 3
PDE Methods for Image Restoration

2. Overview Generic 2nd order nonlinear evolution PDE Classification:
Forward parabolic (smoothing): heat equation
Backward parabolic (smoothing-enhancing): Perona-Malik
Hyperbolic (enhancing): shock-filtering
Artificial time (scales)
Initial degraded image

3. Smoothing PDEs

4. Heat Equation PDE Extend initial value from to
Define space be the extended functions that are integrable on C

5. Heat Equation
Solution

6. Heat Equation
Fourier transform:
Convolution theorem

7. Heat Equation Convolution in Fourier (frequency) domain
Fourier Transform
Attenuating high frequency

8. Heat Equation Convolution in Fourier (frequency) domain
Fourier Transform
Attenuating high frequency

9. Heat Equation Convolution in Fourier (frequency) domain
Fourier Transform
Attenuating high frequency

10. Heat Equation Isotropy. For any two orthogonal directions, we have
The isotropy means that the diffusion is equivalent in the two directions.
In particular

11. Heat Equation Derivation: Let Then
Finally, use the fact that D is unitary

12. Heat Equation

13. Heat Equation
Properties: let , then

14. Heat Equation Properties – continued
All desirable properties for image analysis.
However, edges are smeared out.

15. Nonlinear Diffusion
Introducing nonlinearity hoping for better balance between smoothness and sharpness.
Consider
How to choose the function c(x)? We want:
Smoothing where the norm of gradient is small.
No/Minor smoothing where the norm of gradient is large.

16. Nonlinear Diffusion Impose c(0)=1, smooth and c(x) decreases with x.
Decomposition in normal and tangent direction
Let , then
We impose , which is equivalent to
For example: when s is large

17. Nonlinear Diffusion
How to choose c(x) such that the PDE is well- posed?
Consider
The PDE is parabolic if
Then there exists unique weak solution (under suitable assumptions)

18. Nonlinear Diffusion
How to choose c(x) such that the PDE is well- posed?
Consider
Then, the above PDE is parabolic if we require b(x)>0. (Just need to check det(A)>0 and tr(A)>0.)
where

19. Nonlinear Diffusion Good nonlinear diffusion of the form if Example:

20. The Alvarez–Guichard–Lions–Morel Scale Space Theory
Define a multiscale analysis as a family of operators with
The operator generated by heat equation satisfies a list of axioms that are required for image analysis.
Question: is the converse also true, i.e. if a list of axioms are satisfied, the operator will generate solutions of (nonlinear) PDEs.
More interestingly, can we obtain new PDEs?

21. The Alvarez–Guichard–Lions–Morel Scale Space Theory
Assume the following list of axioms are satisfied

22. The Alvarez–Guichard–Lions–Morel Scale Space Theory

23. The Alvarez–Guichard–Lions–Morel Scale Space Theory

24. The Alvarez–Guichard–Lions–Morel Scale Space Theory
curvature

25. Weickert’s Approach
Motivation: take into account local variations of the gradient orientation.
Observation: is maximal when d is in the same direction as gradient and minimal when its orthogonal to gradient.
Equivalently consider matrix
It has eigenvalues Eigenvectors are in the direction of normal and tangent direction.
It is tempting to define at x an orientation descriptor as a function of

26. Weickert’s Approach Define positive semidefinite matrix
where and is a Gaussian kernel.
Eigenvalues
Classification of structures
Isotropic structures:
Line-like structures:
Corner structures:

27. Weickert’s Approach Nonlinear PDE
Choosing the diffusion tensor D(J): let D(J) have the same eigenvectors as J. Then,
Edge-enhancing anisotropic diffusion
Coherence-enhancing anisotropic diffusion

28. Weickert’s Approach
Edge-enhancing
Original
Processed

29. Weickert’s Approach
Coherence-enhancing
Original
Processed

30. Smoothing-Enhancing PDEs
Perona-Malik Equation

31. The Perona and Malik PDE
Back to general 2nd order nonlinear diffusion
Objective: sharpen edge in the normal direction
Question: how?

32. The Perona and Malik PDE
Idea: backward heat equation.
Recall heat equation and solution
Warning: backward heat equation is ill-posed!
Backward
Forward

33. The Perona and Malik PDE
1D example showing ill-posedness
No classical nor weak solution unless is infinitely differentiable.

34. The Perona and Malik PDE
PM equation
Backward diffusion at edge
Isotropic diffusion at homogeneous regions
Example of such function c(s)
Warning: theoretically solution may not exist.

35. The Perona and Malik PDE

36. The Perona and Malik PDE
Catt′e et al.’s modification

37. Enhancing PDEs
Nonlinear Hyperbolic PDEs (Shock Filters)

38. The Osher and Rudin Shock Filters
A perfect edge
Challenge: go from smooth to discontinuous
Objective: find with edge-sharpening effects

39. The Osher and Rudin Shock Filters
Design of the sharpening PDE (1D): start from

40. The Osher and Rudin Shock Filters
Transport equation (1D constant coefficients)
Variable coefficient transport equation
Example:
Solution:
Solution:

41. The Osher and Rudin Shock Filters
1D design (Osher and Rudin, 1990)
Can we be more precise?

42. Method of Characteristics
Consider a general 1st order PDE
Idea: given an x in U and suppose u is a solution of the above PDE, we would like to compute u(x) by finding some curve lying within U connecting x with a point on Γ and along which we can compute u.
Suppose the curve is parameterized as

43. Method of Characteristics
Define:
Differentiating the second equation of (*) w.r.t. s
Differentiating the original PDE w.r.t.
Evaluating the above equation at x(s)
(*)

44. Method of Characteristics
Letting
Then
Differentiating the first equation of (*) w.r.t. s
Finally
Defines the characteristics
Characteristic ODEs

45. The Osher and Rudin Shock Filters
Consider the simplified PDE with
Convert to the general formulation

46. The Osher and Rudin Shock Filters
First case: Then
and
Thus
For s=0, we have and Thus

47. The Osher and Rudin Shock Filters
Determine :
Since
Using the PDE we have
Thus, we obtain the characteristic curve
and solution
Constant alone characteristic curve

48. The Osher and Rudin Shock Filters
Characteristic curves and solution for case I

49. The Osher and Rudin Shock Filters
Characteristic curves and solution for case II

50. The Osher and Rudin Shock Filters
Observe
Discontinuity (shock) alone the vertical line at
Solution not defined in the white area
To not introduce further discontinuities, we set their values to 1 and -1 respectively
Final solution

51. The Osher and Rudin Shock Filters
More general 1D case
Example
Theoretical guarantee of solutions is still missing
Conjecture by Rudin and Osher

52. The Osher and Rudin Shock Filters
Extension to 2D
Examples of F(s)
Classical
Better
Recall that:

53. The Osher and Rudin Shock Filters
Numerical simulations

54. The Osher and Rudin Shock Filters
Numerical simulations

55. The Osher and Rudin Shock Filters
Drawbacks:
Results obtained are not realistic from a perceptual point of view. Textures will be destroyed.
Noise will be enhanced as well.
Improved version: combining shock filter with anisotropic diffusion

56. Numerical Solutions of PDEs
Finite Difference Approximation

57. Finite Difference Schemes
Solving 1D transport equation

58. Finite Difference Schemes
Solving 1D transport equation
Lax-Friedrichs Scheme
λ=k/h=0.8
Lax-Friedrichs Scheme
λ=k/h=1.6

59. Finite Difference Schemes
Solving 1D transport equation
Lax-Friedrichs Scheme
λ=k/h=0.8
Leapfrog Scheme
λ=k/h=0.8

60. Finite Difference Schemes
Convergence

61. Finite Difference Schemes
Guarantees of convergence: consistency

62. Finite Difference Schemes
Guarantees of convergence: consistency

63. Finite Difference Schemes

64. Finite Difference Schemes
Consistent is necessary but NOT sufficient!

65. Finite Difference Schemes
Consistent is necessary but NOT sufficient!

66. Finite Difference Schemes
Guarantees of convergence: stability
Define
Then (1.5.1) can be written as

67. Finite Difference Schemes
Von Neumann analysis for stability
Fourier series
Inversion
Consider scheme
Plugging in the inversion formula

68. Finite Difference Schemes
Von Neumann analysis for stability
Amplification factor
Important formula
By Parseval’s identity

69. Finite Difference Schemes
Guarantees of convergence: stability
Convergence theorem

70. Finite Difference Schemes
Checking stability

71. Finite Difference Schemes
Heat equation
Standard discretization

72. Finite Difference Schemes
For anisotropic diffusions (e.g. Perona-Malik and Weickert’s equation), we need to approximate
Standard discretization
Not symmetric!

73. Finite Difference Schemes
For anisotropic diffusions (e.g. Perona-Malik and Weickert’s equation), we need to approximate
More symmetric discretization
where

74. Finite Difference Schemes
Shock filters
Discrete approximation
Approximate L using central differencing
Approximating the term using minmod operator
Control Oscillation

75. Homework (Due March 26th 24:00)
Implement heat equation, Perona-Malik equation and shock filters in 2D.
Image restoration problems
Denoising: heat equation and Perona-Malik
Deblurring: shock filters
Observe:
Denoising effects of heat and Perona-Malike, how termination time T affect the results.
How does noise affect deblurring results of shock filters. Compare the two choices of operator L. Observe long term solution.