PDE Methods for Image Restoration Lecture 3 PDE Methods for Image Restoration

Overview Generic 2nd order nonlinear evolution PDE Classification: Forward parabolic (smoothing): heat equation Backward parabolic (smoothing-enhancing): Perona-Malik Hyperbolic (enhancing): shock-filtering Artificial time (scales) Initial degraded image

Smoothing PDEs

Heat Equation PDE Extend initial value from to Define space be the extended functions that are integrable on C

Heat Equation Solution

Heat Equation Fourier transform: Convolution theorem

Heat Equation Convolution in Fourier (frequency) domain Fourier Transform Attenuating high frequency

Heat Equation Convolution in Fourier (frequency) domain Fourier Transform Attenuating high frequency

Heat Equation Convolution in Fourier (frequency) domain Fourier Transform Attenuating high frequency

Heat Equation Isotropy. For any two orthogonal directions, we have The isotropy means that the diffusion is equivalent in the two directions. In particular

Heat Equation Derivation: Let Then Finally, use the fact that D is unitary

Heat Equation

Heat Equation Properties: let , then

Heat Equation Properties – continued All desirable properties for image analysis. However, edges are smeared out.

Nonlinear Diffusion Introducing nonlinearity hoping for better balance between smoothness and sharpness. Consider How to choose the function c(x)? We want: Smoothing where the norm of gradient is small. No/Minor smoothing where the norm of gradient is large.

Nonlinear Diffusion Impose c(0)=1, smooth and c(x) decreases with x. Decomposition in normal and tangent direction Let , then We impose , which is equivalent to For example: when s is large

Nonlinear Diffusion How to choose c(x) such that the PDE is well- posed? Consider The PDE is parabolic if Then there exists unique weak solution (under suitable assumptions)

Nonlinear Diffusion How to choose c(x) such that the PDE is well- posed? Consider Then, the above PDE is parabolic if we require b(x)>0. (Just need to check det(A)>0 and tr(A)>0.) where

Nonlinear Diffusion Good nonlinear diffusion of the form if Example:

The Alvarez–Guichard–Lions–Morel Scale Space Theory Define a multiscale analysis as a family of operators with The operator generated by heat equation satisfies a list of axioms that are required for image analysis. Question: is the converse also true, i.e. if a list of axioms are satisfied, the operator will generate solutions of (nonlinear) PDEs. More interestingly, can we obtain new PDEs?

The Alvarez–Guichard–Lions–Morel Scale Space Theory Assume the following list of axioms are satisfied

The Alvarez–Guichard–Lions–Morel Scale Space Theory

The Alvarez–Guichard–Lions–Morel Scale Space Theory

The Alvarez–Guichard–Lions–Morel Scale Space Theory curvature

Weickert’s Approach Motivation: take into account local variations of the gradient orientation. Observation: is maximal when d is in the same direction as gradient and minimal when its orthogonal to gradient. Equivalently consider matrix It has eigenvalues . Eigenvectors are in the direction of normal and tangent direction. It is tempting to define at x an orientation descriptor as a function of .

Weickert’s Approach Define positive semidefinite matrix where and is a Gaussian kernel. Eigenvalues Classification of structures Isotropic structures: Line-like structures: Corner structures:

Weickert’s Approach Nonlinear PDE Choosing the diffusion tensor D(J): let D(J) have the same eigenvectors as J. Then, Edge-enhancing anisotropic diffusion Coherence-enhancing anisotropic diffusion

Weickert’s Approach Edge-enhancing Original Processed

Weickert’s Approach Coherence-enhancing Original Processed

Smoothing-Enhancing PDEs Perona-Malik Equation

The Perona and Malik PDE Back to general 2nd order nonlinear diffusion Objective: sharpen edge in the normal direction Question: how?

The Perona and Malik PDE Idea: backward heat equation. Recall heat equation and solution Warning: backward heat equation is ill-posed! Backward Forward

The Perona and Malik PDE 1D example showing ill-posedness No classical nor weak solution unless is infinitely differentiable.

The Perona and Malik PDE PM equation Backward diffusion at edge Isotropic diffusion at homogeneous regions Example of such function c(s) Warning: theoretically solution may not exist.

The Perona and Malik PDE

The Perona and Malik PDE Catt′e et al.’s modification

Enhancing PDEs Nonlinear Hyperbolic PDEs (Shock Filters)

The Osher and Rudin Shock Filters A perfect edge Challenge: go from smooth to discontinuous Objective: find with edge-sharpening effects

The Osher and Rudin Shock Filters Design of the sharpening PDE (1D): start from

The Osher and Rudin Shock Filters Transport equation (1D constant coefficients) Variable coefficient transport equation Example: Solution: Solution:

The Osher and Rudin Shock Filters 1D design (Osher and Rudin, 1990) Can we be more precise?

Method of Characteristics Consider a general 1st order PDE Idea: given an x in U and suppose u is a solution of the above PDE, we would like to compute u(x) by finding some curve lying within U connecting x with a point on Γ and along which we can compute u. Suppose the curve is parameterized as

Method of Characteristics Define: Differentiating the second equation of (*) w.r.t. s Differentiating the original PDE w.r.t. Evaluating the above equation at x(s) (*)

Method of Characteristics Letting Then Differentiating the first equation of (*) w.r.t. s Finally Defines the characteristics Characteristic ODEs

The Osher and Rudin Shock Filters Consider the simplified PDE with Convert to the general formulation

The Osher and Rudin Shock Filters First case: . Then and Thus For s=0, we have and . Thus

The Osher and Rudin Shock Filters Determine : Since Using the PDE we have Thus, we obtain the characteristic curve and solution Constant alone characteristic curve

The Osher and Rudin Shock Filters Characteristic curves and solution for case I

The Osher and Rudin Shock Filters Characteristic curves and solution for case II

The Osher and Rudin Shock Filters Observe Discontinuity (shock) alone the vertical line at Solution not defined in the white area To not introduce further discontinuities, we set their values to 1 and -1 respectively Final solution

The Osher and Rudin Shock Filters More general 1D case Example Theoretical guarantee of solutions is still missing Conjecture by Rudin and Osher

The Osher and Rudin Shock Filters Extension to 2D Examples of F(s) Classical Better Recall that:

The Osher and Rudin Shock Filters Numerical simulations

The Osher and Rudin Shock Filters Numerical simulations

The Osher and Rudin Shock Filters Drawbacks: Results obtained are not realistic from a perceptual point of view. Textures will be destroyed. Noise will be enhanced as well. Improved version: combining shock filter with anisotropic diffusion

Numerical Solutions of PDEs Finite Difference Approximation

Finite Difference Schemes Solving 1D transport equation

Finite Difference Schemes Solving 1D transport equation Lax-Friedrichs Scheme λ=k/h=0.8 Lax-Friedrichs Scheme λ=k/h=1.6

Finite Difference Schemes Solving 1D transport equation Lax-Friedrichs Scheme λ=k/h=0.8 Leapfrog Scheme λ=k/h=0.8

Finite Difference Schemes Convergence

Finite Difference Schemes Guarantees of convergence: consistency

Finite Difference Schemes Guarantees of convergence: consistency

Finite Difference Schemes

Finite Difference Schemes Consistent is necessary but NOT sufficient!

Finite Difference Schemes Consistent is necessary but NOT sufficient!

Finite Difference Schemes Guarantees of convergence: stability Define Then (1.5.1) can be written as

Finite Difference Schemes Von Neumann analysis for stability Fourier series Inversion Consider scheme Plugging in the inversion formula

Finite Difference Schemes Von Neumann analysis for stability Amplification factor Important formula By Parseval’s identity

Finite Difference Schemes Guarantees of convergence: stability Convergence theorem

Finite Difference Schemes Checking stability

Finite Difference Schemes Heat equation Standard discretization

Finite Difference Schemes For anisotropic diffusions (e.g. Perona-Malik and Weickert’s equation), we need to approximate Standard discretization Not symmetric!

Finite Difference Schemes For anisotropic diffusions (e.g. Perona-Malik and Weickert’s equation), we need to approximate More symmetric discretization where

Finite Difference Schemes Shock filters Discrete approximation Approximate L using central differencing Approximating the term using minmod operator Control Oscillation

Homework (Due March 26th 24:00) Implement heat equation, Perona-Malik equation and shock filters in 2D. Image restoration problems Denoising: heat equation and Perona-Malik Deblurring: shock filters Observe: Denoising effects of heat and Perona-Malike, how termination time T affect the results. How does noise affect deblurring results of shock filters. Compare the two choices of operator L. Observe long term solution.