L ECTURE 3 PDE Methods for Image Restoration

O VERVIEW Generic 2 nd order nonlinear evolution PDE Classification: Forward parabolic (smoothing): heat equation Backward parabolic (smoothing-enhancing): TV flow Hyperbolic (enhancing): shock-filtering Initial degraded image Artificial time (scales)

S MOOTHING PDE S

H EAT E QUATION PDE Extend initial value from to Define space be the extended functions that are integrable on C

H EAT E QUATION Solution

H EAT E QUATION Fourier transform: Convolution theorem

H EAT E QUATION Convolution in Fourier (frequency) domain Fourier TransformAttenuating high frequency

H EAT E QUATION Convolution in Fourier (frequency) domain Fourier TransformAttenuating high frequency

H EAT E QUATION Convolution in Fourier (frequency) domain Fourier TransformAttenuating high frequency

H EAT E QUATION Isotropy. For any two orthogonal directions, we have The isotropy means that the diffusion is equivalent in the two directions. In particular

H EAT E QUATION Derivation: Let Then Finally, use the fact that D is unitary

H EAT E QUATION

Properties: let, then

H EAT E QUATION Properties – continued All desirable properties for image analysis. However, edges are smeared out.

N ONLINEAR D IFFUSION Introducing nonlinearity hoping for better balance between smoothness and sharpness. Consider How to choose the function c(x)? We want: Smoothing where the norm of gradient is small. No/Minor smoothing where the norm of gradient is large.

N ONLINEAR D IFFUSION Decomposition in normal and tangent direction Let, then We impose, which is equivalent to For example: when s is large

N ONLINEAR D IFFUSION Well-posed? The PDE is parabolic if It reduces to where

N ONLINEAR D IFFUSION Good nonlinear diffusion of the form if Example: or

T HE A LVAREZ –G UICHARD –L IONS –M OREL S CALE S PACE T HEORY Define a multiscale analysis as a family of operators with The operator generated by heat equation satisfies a list of axioms that are required for image analysis. Question: is the converse also true, i.e. if a list of axioms are satisfied, the operator will generate solutions of (nonlinear) PDEs. More interestingly, can we obtain new PDEs?

T HE A LVAREZ –G UICHARD –L IONS –M OREL S CALE S PACE T HEORY Assume the following list of axioms are satisfied

T HE A LVAREZ –G UICHARD –L IONS –M OREL S CALE S PACE T HEORY

curvature

W EICKERT ’ S A PPROACH Motivation: take into account local variations of the gradient orientation. Observation: is maximal when d is in the same direction as gradient and minimal when its orthogonal to gradient. Equivalently consider matrix It has eigenvalues. Eigenvectors are in the direction of normal and tangent direction.

W EICKERT ’ S A PPROACH Define positive semidefinite matrix where and is a Gaussian kernel. Eigenvalues Classification of structures Isotropic structures: Line-like structures: Corner structures:

W EICKERT ’ S A PPROACH Nonlinear PDE Choosing the diffusion tensor D(J): let D(J) have the same eigenvectors as J. Then, Edge-enhancing anisotropic diffusion Coherence-enhancing anisotropic diffusion

W EICKERT ’ S A PPROACH Edge-enhancing OriginalProcessed

W EICKERT ’ S A PPROACH Coherence-enhancing OriginalProcessed

S MOOTHING -E NHANCING PDE S Perona-Malik Equation

T HE P ERONA AND M ALIK PDE Back to general 2 nd order nonlinear diffusion Objective: sharpen edge in the normal direction Question: how?

T HE P ERONA AND M ALIK PDE Idea: backward heat equation. Recall heat equation and solution Warning: backward heat equation is ill-posed! BackwardForward

T HE P ERONA AND M ALIK PDE 1D example showing ill-posedness No classical nor weak solution unless is infinitely differentiable.

T HE P ERONA AND M ALIK PDE PM equation Backward diffusion at edge Isotropic diffusion at homogeneous regions Example of such function c(s) Warning: theoretically solution may not exist.

T HE P ERONA AND M ALIK PDE

Catt′e et al.’s modification

E NHANCING PDE S Nonlinear Hyperbolic PDEs (Shock Filters)

T HE O SHER AND R UDIN S HOCK F ILTERS A perfect edge Challenge: go from smooth to discontinuous Objective: find with edge-sharpening effects

T HE O SHER AND R UDIN S HOCK F ILTERS Design of the sharpening PDE (1D): start from

T HE O SHER AND R UDIN S HOCK F ILTERS Transport equation (1D constant coefficients) Variable coefficient transport equation Example: Solution:

T HE O SHER AND R UDIN S HOCK F ILTERS 1D design (Osher and Rudin, 1990) Can we be more precise?

M ETHOD OF C HARACTERISTICS Consider a general 1 st order PDE Idea: given an x in U and suppose u is a solution of the above PDE, we would like to compute u(x) by finding some curve lying within U connecting x with a point on Γ and along which we can compute u. Suppose the curve is parameterized as

M ETHOD OF C HARACTERISTICS Define: Differentiating the second equation of (*) w.r.t. s Differentiating the original PDE w.r.t. Evaluating the above equation at x(s) (*)

M ETHOD OF C HARACTERISTICS Letting Then Differentiating the first equation of (*) w.r.t. s Finally Characteristic ODEs

T HE O SHER AND R UDIN S HOCK F ILTERS Consider the simplified PDE with Convert to the general formulation

T HE O SHER AND R UDIN S HOCK F ILTERS First case:. Then and Thus For s=0, we have and. Thus

T HE O SHER AND R UDIN S HOCK F ILTERS Determine : o Since o Using the PDE we have Thus, we obtain the characteristic curve and solution Constant alone characteristic curve

T HE O SHER AND R UDIN S HOCK F ILTERS Characteristic curves and solution for case I

T HE O SHER AND R UDIN S HOCK F ILTERS Characteristic curves and solution for case II

T HE O SHER AND R UDIN S HOCK F ILTERS Observe o Discontinuity (shock) alone the vertical line at o Solution not defined in the gray areas o To not introduce further discontinuities, we set their values to 1 and -1 respectively Final solution

T HE O SHER AND R UDIN S HOCK F ILTERS Extension to 2D Examples of F(s) o Classical o Better 2 nd order differentiation in normal direction

T HE O SHER AND R UDIN S HOCK F ILTERS Numerical simulations

T HE O SHER AND R UDIN S HOCK F ILTERS Numerical simulations

T HE O SHER AND R UDIN S HOCK F ILTERS Drawbacks: o Results obtained are not realistic from a perceptual point of view. Textures will be destroyed. o Noise will be enhanced as well. Improved version: combining shock filter with anisotropic diffusion

N UMERICAL S OLUTIONS OF PDE S Finite Difference Approximation

F INITE D IFFERENCE S CHEMES Discretization of computation domain (1D) Basic discretizations of first order derivatives

F INITE D IFFERENCE S CHEMES Heat equation Standard discretization Boundary condition: If the value of a pixel (vertex) that is outside the domain is needed, we use the value of the pixel that is symmetric with respect to the boundaries.

F INITE D IFFERENCE S CHEMES For anisotropic diffusions (e.g. Perona-Malik and Weickert’s equation), we need to approximate Standard discretization Not symmetric!

F INITE D IFFERENCE S CHEMES For anisotropic diffusions (e.g. Perona-Malik and Weickert’s equation), we need to approximate More symmetric discretization where

F INITE D IFFERENCE S CHEMES Shock filters Discrete approximation o Approximate L using central differencing o Approximating the term using minmod operator Control Oscillation

H OMEWORK (D UE A PRIL 13 TH 11:59 PM ) Implement heat equation, Perona-Malik equation and shock filters in 2D. Image restoration problems o Denoising: heat equation and Perona-Malik o Deblurring: shock filters Observe: o Denoising effects of heat and Perona-Malike, how termination time T affect the results. o How does noise affect deblurring results of shock filters. Compare the two choices of operator L. Observe long term solution.