1. 1 Regularisierung mir Singulären Energien Martin Burger Institut für Numerische und Angewandte Mathematik Westfälische Wilhelms Universität Münster martin.burger@uni-muenster.de TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A AA

2. Regularisierung mit singulären Energien Göttingen, Januar 20072 Stan Osher, Jinjun Xu, Guy Gilboa (UCLA) Lin He (Linz / UCLA) Klaus Frick, Otmar Scherzer (Innsbruck) Carola Schönlieb (Vienna) Don Goldfarb, Wotao Yin (Columbia) Collaborations

3. Regularisierung mit singulären Energien Göttingen, Januar 20073 Classical regularization schemes for inverse problems and image smoothing are based on Hilbert spaces and quadratic energy functionals Example: Tikhonov regularization for linear operator equations Introduction ¸ 2 k A u ¡ f k 2 + 1 2 k L u k 2 ! m i n u

4. Regularisierung mit singulären Energien Göttingen, Januar 20074 These energy functionals are strictly convex and differentiable – standard tools from analysis and computation (Newton methods etc.) can be used Disadvantage: possible oversmoothing, seen from first-order optimality condition Tikhonov yields Hence u is in the range of (L*L) -1 A* Introduction L ¤ L u = ¡ ¸ A ¤ ( A u f )

5. Regularisierung mit singulären Energien Göttingen, Januar 20075 Classical inverse problem: integral equation of the first kind, regularization in L 2 (L = Id), A = Fredholm integral operator with kernel k Smoothness of regularized solution is determined by smoothness of kernel For typical convolution kernels like Gaussians, u is analytic ! Introduction u = ¸ ZZ k ( y ; x )( ¡ k ( y ; z ) u ( z ) + f ( z )) d y d z

6. Regularisierung mit singulären Energien Göttingen, Januar 20076 Classical image smoothing: data in L 2 (A = Id), L = gradient (H 1 -Seminorm) On a reasonable domain, standard elliptic regularity implies Reconstruction contains no edges, blurs the image (with Green kernel) Image Smoothing ¡ ¢ u + ¸ u = ¸f u 2 H 2 ( ­ ), ! C ( ­ )

7. Regularisierung mit singulären Energien Göttingen, Januar 20077 Let A be an operator on (basis repre- sentation of a Hilbert space operator, wavelet) Penalization by squared norm (L = Id) Optimality condition for components of u Decay of components determined by A*. Even if data are generated by sparse signal (finite number of nonzeros), reconstruction is not sparse ! Sparse Reconstructions ? ` 2 ( Z ) u k = ¸ ( A ¤ ( ¡ A u + f )) k

8. Regularisierung mit singulären Energien Göttingen, Januar 20078 Error estimates for ill-posed problems can be obtained only under stronger conditions (source conditions) cf. Groetsch, Engl-Hanke-Neubauer, Colton-Kress, Natterer. Engl-Kunisch-Neubauer. Equivalent to u being minimizer of Tikhonov functional with data For many inverse problems unrealistic due to extreme smoothness assumptions Error estimates 9 w:u = A ¤ w

9. Regularisierung mit singulären Energien Göttingen, Januar 20079 Condition can be weakened to cf. Neubauer et al (algebraic), Hohage (logarithmic), Mathe-Pereverzyev (general). Advantage: more realistic conditions Disadvantage: Estimates get worse with f Error estimates 9 v:u = f ( A ¤ A ) v

10. Regularisierung mit singulären Energien Göttingen, Januar 200710 Let A be the identity on Nonlinear Penalization by Optimality condition for components of u If r k is smooth and strictly convex, then Taylor expansion yields Singular Energies ` 2 ( Z ) P r k ( u k ) r 00 k ( f k ) u k + ¸ u k ¼ r 00 k ( f k ) f k + ¸f k r 0 k ( u k ) + ¸ u k = ¸f k

11. Regularisierung mit singulären Energien Göttingen, Januar 200711 Example becomes more interesting for singular (nonsmooth) energy Take Then optimality condition becomes Singular Energies r k ( t ) = j t j s i gn ( u k ) + ¸ u k = ¸f k

12. Regularisierung mit singulären Energien Göttingen, Januar 200712 Result is well-known soft-thresholding of wavelets Donoho et al, Chambolle et al Yields a sparse signal Singular Energies u k = 8 < : f k ¡ 1 ¸ f k > 1 ¸ f k + 1 ¸ f k < ¡ 1 ¸ 0 e l se

13. Regularisierung mit singulären Energien Göttingen, Januar 200713 Image smoothing: try nonlinear energy for penalization Optimality condition is nonlinear PDE If r is strictly convex usual smoothing behaviour If r is not convex problem not well-posed Try singular case at the borderline Singular Energies Z r ( r u ) ¡ r ¢ (( r r )( r u )) + ¸ u = ¸f

14. Regularisierung mit singulären Energien Göttingen, Januar 200714 Simplest choice yields total variation method Total variation methods are popular in imaging (and inverse problems), since - they keep sharp edges - eliminate oscillations (noise) - create new nice mathematics Total Variation Methods r ( p ) = j p j

15. Regularisierung mit singulären Energien Göttingen, Januar 200715 ROF model for denoising Rudin-Osher Fatemi 89/92, Acar-Vogel 93, Chambolle-Lions 96, Vogel 95/96, Scherzer-Dobson 96, Chavent-Kunisch 98, Meyer 01,… ROF Model

16. Regularisierung mit singulären Energien Göttingen, Januar 200716 Optimality condition for ROF denoising Dual variable p enters ! Subgradient of convex functional ROF Model p + ¸ u = ¸f ; p 2 @ j u j TV @ J ( u ) = f p 2 X ¤ j 8 v 2 X : J ( u ) ¡ h p ; v ¡ u i · J ( v ) g

17. Regularisierung mit singulären Energien Göttingen, Januar 200717 ROF Model Reconstruction (code by Jinjun Xu) cleannoisy ROF

18. Regularisierung mit singulären Energien Göttingen, Januar 200718 ROF model denoises cartoon images resp. computes the cartoon of an arbitrary image ROF Model

19. Regularisierung mit singulären Energien Göttingen, Januar 200719 From Master Thesis of Markus Bachmayr, 2007 Numerical Differentiation with TV

20. Regularisierung mit singulären Energien Göttingen, Januar 200720 Methods with singular energies offer great potential, but still have some shortcomings - difficult to analyze and to obtain error estimates - systematic errors (clean images not reconstructed perfectly) - computational challenges - some extensions to complicated imaging tasks are not well understood (e.g. inpainting) Singular energies

21. Regularisierung mit singulären Energien Göttingen, Januar 200721 General problem leads to optimality condition First of all „dual smoothing“, subgradient p is in the range of A* Singular energies ¸ 2 k A u ¡ f k 2 + J ( u ) ! m i n u p + ¸ A ¤ A u = ¸ A ¤ f ; p 2 @ J ( u )

22. Regularisierung mit singulären Energien Göttingen, Januar 200722 For smooth and strictly convex energies, the subdifferential is a singleton Dual smoothing directly results in a primal one ! For singular energies, subdifferentials are not usually multivalued. The consequence is a possibility to break the primal smoothing Singular energies @ J ( u ) = f J 0 ( u ) g

23. Regularisierung mit singulären Energien Göttingen, Januar 200723 First question for error estimation: estimate difference of u (minimizer of ROF) and f in terms of Estimate in the L 2 norm is standard, but does not yield information about edges Estimate in the BV-norm too ambitious: even arbitrarily small difference in edge location can yield BV-norm of order one ! Error Estimation

24. Regularisierung mit singulären Energien Göttingen, Januar 200724 We need a better error measure, stronger than L 2, weaker than BV Possible choice: Bregman distance Bregman 67 Real distance for a strictly convex differentiable functional – not symmetric Symmetric version Error Estimation

25. Regularisierung mit singulären Energien Göttingen, Januar 200725 Bregman distances reduce to known measures for standard energies Example 1: Subgradient = Gradient = u Bregman distance becomes Error Estimation J ( u ) = 1 2 k u k 2 D J ( u ; v ) = 1 2 k u ¡ v k 2

26. Regularisierung mit singulären Energien Göttingen, Januar 200726 Bregman distances reduce to known measures for standard energies Example 2: - Subgradient = Gradient = log u Bregman distance becomes Kullback-Leibler divergence (relative Entropy) Error Estimation J ( u ) = Z u l ogu Z u D J ( u ; v ) = Z u l og u v + Z ( v ¡ u )

27. Regularisierung mit singulären Energien Göttingen, Januar 200727 Total variation is neither symmetric nor differentiable Define generalized Bregman distance for each subgradient Symmetric version Kiwiel 97, Chen-Teboulle 97 Error Estimation

28. Regularisierung mit singulären Energien Göttingen, Januar 200728 For energies homogeneous of degree one, we have Bregman distance becomes Error Estimation

29. Regularisierung mit singulären Energien Göttingen, Januar 200729 Bregman distance for singular energies is not a strict distance, can be zero for In particular d TV is zero for contrast change Resmerita-Scherzer 06 Bregman distance is still not negative (convexity) Bregman distance can provide information about edges Error Estimation

30. Regularisierung mit singulären Energien Göttingen, Januar 200730 Let v be piecewise constant with white background and color values on regions Then we obtain subgradients of the form with signed distance function and Error Estimation

31. Regularisierung mit singulären Energien Göttingen, Januar 200731 Bregman distances given by In the limit we obtain for being piecewise continuous Error Estimation

32. Regularisierung mit singulären Energien Göttingen, Januar 200732 For estimate in terms of we need smoothness condition on data Optimality condition for ROF Error Estimation

33. Regularisierung mit singulären Energien Göttingen, Januar 200733 Subtract q Estimate for Bregman distance, mb-Osher 04 Error Estimation

34. Regularisierung mit singulären Energien Göttingen, Januar 200734 In practice we have to deal with noisy data f (perturbation of some exact data g) Estimate for Bregman distance Error Estimation

35. Regularisierung mit singulären Energien Göttingen, Januar 200735 Optimal choice of the penalization parameter i.e. of the order of the noise variance Error Estimation

36. Regularisierung mit singulären Energien Göttingen, Januar 200736 Direct extension to deconvolution / linear inverse problems under standard source condition mb-Osher 04 Extension: stronger estimates under stronger conditions, Resmerita 05 Nonlinear inverse problems, Resmerita-Scherzer 06 Error Estimation ¸ 2 k A u ¡ f k 2 + j u j TV ! m i n u 2 BV

37. Regularisierung mit singulären Energien Göttingen, Januar 200737 Natural choice: primal discretization with piecewise constant functions on grid Problem 1: Numerical analysis (characterization of discrete subgradients) Problem 2: Discrete problems are the same for any anisotropic version of the total variation Discretization

38. Regularisierung mit singulären Energien Göttingen, Januar 200738 In multiple dimensions, nonconvergence of the primal discretization for the isotropic TV (p=2) can be shown Convergence of anisotropic TV (p=1) on rectangular aligned grids Fitzpatrick-Keeling 1997 Discretization

39. Regularisierung mit singulären Energien Göttingen, Januar 200739 Alternative: perform primal-dual discretization for optimality system (variational inequality) with convex set Primal-Dual Discretization

40. Regularisierung mit singulären Energien Göttingen, Januar 200740 Discretization Discretized convex set with appropriate elements (piecewise linear in 1D, Raviart- Thomas in multi-D) Primal-Dual Discretization

41. Regularisierung mit singulären Energien Göttingen, Januar 200741 In 1 D primal, primal-dual, and dual discretization are equivalent Error estimate for Bregman distance by analogous techniques Note that only the natural condition is needed to show Primal / Primal-Dual Discretization

42. Regularisierung mit singulären Energien Göttingen, Januar 200742 In multi-D similar estimates, additional work since projection of subgradient is not discrete subgradient. Primal-dual discretization equivalent to discretized dual minimization (Chambolle 03, Kunisch-Hintermüller 04). Can be used for existence of discrete solution, stability of p Mb 07 ? Primal / Primal-Dual Discretization

43. Regularisierung mit singulären Energien Göttingen, Januar 200743 For most imaging applications Cartesian grids are used. Primal dual discretization can be reinterpreted as a finite difference scheme in this setup. Value of image intensity corresponds to color in a pixel of width h around the grid point. Raviart-Thomas elements on Cartesian grids particularly easy. First component piecewise linear in x, pw constant in y,z, etc. Leads to simple finite difference scheme with staggered grid Cartesian Grids

44. Regularisierung mit singulären Energien Göttingen, Januar 200744 ROF minimization has a systematic error, total variation of the reconstruction is smaller than total variation of clean image. Image features left in residual f-u g, clean f, noisy u, ROFf-u Iterative Refinement & ISS

45. Regularisierung mit singulären Energien Göttingen, Januar 200745 Idea: add the residual („noise“) back to the image to pronounce the features decreased to much. Then do ROF again. Iterative procedure Osher-mb-Goldfarb-Xu-Yin 04 Iterative Refinement & ISS

46. Regularisierung mit singulären Energien Göttingen, Januar 200746 Improves reconstructions significantly Iterative Refinement & ISS

47. Regularisierung mit singulären Energien Göttingen, Januar 200747 Iterative Refinement & ISS

48. Regularisierung mit singulären Energien Göttingen, Januar 200748 Simple observation from optimality condition Consequently, iterative refinement equivalent to Bregman iteration Iterative Refinement & ISS

49. Regularisierung mit singulären Energien Göttingen, Januar 200749 Choice of parameter less important, can be kept small (oversmoothing). Regularizing effect comes from appropriate stopping. Quantitative stopping rules available, or „stop when you are happy“ – S.O. Limit to zero can be studied. Yields gradient flow for the dual variable („inverse scale space“) mb-Gilboa-Osher-Xu 06, mb-Frick-Osher-Scherzer 06 Iterative Refinement & ISS

50. Regularisierung mit singulären Energien Göttingen, Januar 200750 Non-quadratic fidelity is possible, some caution needed for L 1 fidelity He-mb-Osher 05, mb-Frick-Osher-Scherzer 06 Error estimation in Bregman distance mb-He-Resmerita 07 Iterative Refinement & ISS

51. Regularisierung mit singulären Energien Göttingen, Januar 200751 MRI Data Siemens Magnetom Avanto 1.5 T Scanner He, Chang, Osher, Fang, Speier 06 PenalizationTV + Wavelet Iterative Refinement

52. Regularisierung mit singulären Energien Göttingen, Januar 200752 MRI Data Siemens Magnetom Avanto 1.5 T Scanner He, Chang, Osher, Fang, Speier 06 Iterative Refinement

53. Regularisierung mit singulären Energien Göttingen, Januar 200753 MRI Data Siemens Magnetom Avanto 1.5 T Scanner He, Chang, Osher, Fang, Speier 06 Iterative Refinement

54. Regularisierung mit singulären Energien Göttingen, Januar 200754 Smoothing of surfaces obtained as level sets 3D Ultrasound, Kretz / GE Med. Surface Smoothing

55. Regularisierung mit singulären Energien Göttingen, Januar 200755 Inverse Scale Space

56. Regularisierung mit singulären Energien Göttingen, Januar 200756 Application to other regularization techniques, e.g. wavelet thresholding is straightforward Starting from soft shrinkage, iterated refinement yields firm shrinkage, inverse scale space becomes hard shrinkage Osher-Xu 06 Bregman distance natural sparsity measure, source condition just requires sparse signal, number of nonzero components is smoothness measure in error estimates Iterative Refinement & ISS

57. Regularisierung mit singulären Energien Göttingen, Januar 200757 Difficult to construct total variation techniques for inpainting Original extensions of ROF failed to obtain natural connectivity (see book by Chan, Shen 05) Inpainting region, image f (noisy) given on Try to minimize Inpainting

58. Regularisierung mit singulären Energien Göttingen, Januar 200758 Optimality condition will have the form with A being a linear operator defining the norm In particular p = 0 in D ! Inpainting

59. Regularisierung mit singulären Energien Göttingen, Januar 200759 Different iterated approach (motivated by Cahn-Hilliard inpainting, Bertozzi et al 05 ) Minimize in each step First term for damping, second for fidelity (fit to f where given, and to old iterate in the inpainting region), third term for smoothing Inpainting

60. Regularisierung mit singulären Energien Göttingen, Januar 200760 Continuous flow for damping parameter to zero Fourth order flow for H -1 norm Stationary solution (existence ?) satisfies Inpainting

61. Regularisierung mit singulären Energien Göttingen, Januar 200761 Result: Penguins Inpainting

62. Regularisierung mit singulären Energien Göttingen, Januar 200762 Download and Contact Papers and Talks: www.math.uni-muenster.de/u/burger e-mail: martin.burger@uni-muenster.de