Presentation is loading. Please wait.

Presentation is loading. Please wait.

DESIGN IN IMAGING FROM COMPRESSIVE TO COMPREHENSIVE SENSING Lior Horesh Joint work with E. Haber, L. Tenorio, A. Conn, U. Mello, J. Fohring IBM TJ Watson.

Similar presentations


Presentation on theme: "DESIGN IN IMAGING FROM COMPRESSIVE TO COMPREHENSIVE SENSING Lior Horesh Joint work with E. Haber, L. Tenorio, A. Conn, U. Mello, J. Fohring IBM TJ Watson."— Presentation transcript:

1 DESIGN IN IMAGING FROM COMPRESSIVE TO COMPREHENSIVE SENSING Lior Horesh Joint work with E. Haber, L. Tenorio, A. Conn, U. Mello, J. Fohring IBM TJ Watson Research Center SIAM Imaging Science 2012, Philadelphia, PA - May 2012

2 INTRODUCTION

3 Aim: infer model Given Experimental design y Measurements Observation model Ill - posedness  Naïve inversion... Fails... Need to fill in the missing information EXPOSITION - ILL - POSED INVERSE PROBLEMS

4 PROBLEM CHARACTERISTICS Indirect data (e.g. observation operator non-linear, based on PDEs) Noisy Incomplete Ill-posed Large-scale

5 HOW TO IMPROVE MODEL RECOVERY ? How can we... Improve observation model ? Incorporate more meaningful a-priori information ? Extract more information in the measurement procedure ? Provide more efficient optimization schemes ? Define appropriate distance measures / noise model ?

6 Forward problem (simulation, description) Given: model m & observation model Simulate: data d Inverse problem (estimation, prediction) Given: data d & observation model Infer: model m (and uncertainties) Design (prescription) Given: inversion scheme & observation model Find: the ‘best’ experimental settings y, regularization S, … EVOLUTION OF INVERSION FROM SIMULATION TO DESIGN

7 WHY DESIGN ?

8 WHICH EXPERIMENTAL DESIGN IS BEST ? Design 1Design 3Design 2

9 Where light sources and optodes should be placed ? In what sequence they should be activated ? What laser source intensities should be used ? DESIGN QUESTIONS – DIFFUSE OPTICAL TOMOGRAPHY Arridge 1999

10 What frequencies should be used ? What trajectory of acquisition should be considered ? DESIGN QUESTIONS – ELECTROMAGNETIC INVERSION Newman 1996, Haber & Ascher 2000

11 DESIGN QUESTIONS – SEISMIC INVERSION Clerbout 2000 How simultaneous sources can be used ?

12 DESIGN QUESTIONS – ULTRASOUND IMAGING What levels of ultrasonic waves should be used ? Ludwig 1949

13 How many projections should we collect and at what angles ? How accurate should the projection data be ? DESIGN QUESTIONS – LIMITED ANGLE TOMOGRAPHY

14 WHAT CAN WE DESIGN ?

15 DESIGN EXPERIMENTAL LAYOUT Stonehenge 2500 B.C.

16 DESIGN EXPERIMENTAL PROCESS Galileo Galilei

17 RESPECT EXPERIMENTAL CONSTRAINTS… French nuclear test, Mururoa, 1970

18 HOW TO DESIGN ?

19 THE PHILOSOPHY The goal in designing an experiment is to maximize the expected utility that is chosen to reflect purposes of the experiment The utility is usually defined by measuring the accuracy and fidelity of the information provided by the experiments

20 Previous work Well-posed problems - well established [ Fedorov 1997, Pukelsheim 2006 ] Ill-posed problems - under-researched [ Curtis 1999, Bardow 2008 ] Many practical problems in engineering and sciences are ill-posed What makes ill-posed problems so special ? ILL VS. WELL - POSED OPTIMAL EXPERIMENTAL DESIGN

21 OPTIMALITY CRITERIA IN WELL-POSED PROBLEMS For linear inversion, employ Tikhonov regularized least squares solution Bias - variance decomposition For over-determined problems A-optimal design problem

22 OPTIMALITY CRITERIA IN WELL-POSED PROBLEMS Optimality criteria of the information matrix A-optimal design  average variance D-optimality  uncertainty ellipsoid E-optimality  minimax Almost a complete alphabet… Stone, DeGroot and Bernardo

23 THE PROBLEMS... Ill-posedness  controlling variance alone reduces the error mildly [ Johansen 1996 ] Non-linearity  bias-variance decomposition is impossible What strategy can be used ? Proposition 1 - Common practice so far Trial and Error…

24 EXPERIMENTAL DESIGN BY TRIAL AND ERROR Pick a model Run observation model of different experimental designs, and get data Invert and compare recovered models Choose the experimental design that provides the best model recovery

25 THE PROBLEMS... Ill-posedness  controlling variance alone reduces the error mildly [ Johansen 1996 ] Non-linearity  bias-variance decomposition is impossible What other strategy can be used ? Proposition 2 - Minimize bias and variance altogether How to define the optimality criterion ?

26 OPTIMALITY CRITERIA FOR DESIGN Loss Mean Square Error Bayes risk  Depends on the noise   Depends on unknown model  Computationally infeasible

27 OPTIMALITY CRITERIA FOR DESIGN Bayes empirical risk Assume a set of authentic model examples is available Discrepancy between training and recovered models [ Vapnik 1998 ] How can y and S be regularized ? Regularized Bayesian empirical risk

28 OTHER DESIGNERS / KEY PLAYERS This is one doctrine Other interesting choices were developed by Y. Marzouk et. al, MIT (2011, 2012) A. Curtis et. al, University of Edinburgh (1999, 2010) D. Coles & M. Prange, Schlumberger (2008, 2012) S. Körkel et. al, Heidelberg University (2011) A. Bardow, RWTH Aachen University (2008, 2009)

29 DIFFERENTIABLE OBSERVATION SPARSITY CONTROLLED DESIGN Assume: fixed number of observations Design preference: small number of sources / receivers is activated The observation model Regularized risk Horesh, Haber & Tenorio 2011

30 DIFFERENTIABLE OBSERVATION SPARSITY CONTROLLED DESIGN Total number of observations may be large Derivatives of the forward operator w.r.t. y Effective when activation of each source and receiver is expensive Horesh, Haber & Tenorio 2011 Difficult…

31 WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN Assume: a predefined set of candidate experimental setups is given Design preference: small number of observations Haber, Horesh & Tenorio 2010

32 WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN Let be discretization of the space [ Pukelsheim 1994 ] Let The (candidates) observation operator is weighted w inverse of standard deviation - reasonable standard deviation - conduct the experiment - infinite standard deviation - do not conduct the experiment Haber, Horesh & Tenorio 2010

33 WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN Solution - more observations give better recovery Desired solution many w‘s are 0 Add penalty to promote sparsity in observations selection Less degrees of freedom No explicit access to the observation operator needed

34 THE OPTIMIZATION PROBLEMS Leads to a stochastic bi-level optimization problem Direct formulation Weights formulation Haber, Horesh & Tenorio 2010, 2011

35 THE OPTIMIZATION PROBLEM(S) In general (multi-level) optimization problem Difficult to solve (although not impossible) [ Alexaderov & Denis 1994 ] To simplify make assumptions on F - linear, nonlinear S - quadratic, matrix form y - discrete, continuous Important - sufficient improvement can be “good enough”

36 HOW MANY SAMPLES ARE NEEDED ? The optimist - choose a single m and design for it The pessimist - assume that m is the worst it can be The realist - choose an ensemble of m’s and average A Bayesian estimate of the frequentist risk [ Rubin 1984 ]

37 HOW MANY SAMPLES ARE NEEDED ? What type of character are you ? The Optimist The Realist The Pessimist

38 OPTIMAL DESIGN IN PRACTICE

39 LINEAR DESIGN

40 Assume Then

41 LINEAR DESIGN Assume m has second moment Then Best linear design

42 CASE STUDY I LINEAR EXPERIMENTAL DESIGN

43 THE BEST (LINEAR) EXPERIMENT Considering discretization of the design space We obtain

44 BOREHOLE RAY TOMOGRAPHY - OBSERVATION MODEL Use 32 2 × 3 = 3072 rays Goal: choose 500 optimal observations

45 BOREHOLE RAY TOMOGRAPHY EXPERIMENTAL DESIGN ASSESSMENT Assess design performance upon unseen models (cross validation) Test modelOptimal designNon-optimal design

46 NON - LINEAR DESIGN

47 Use Stochastic Optimization - approximate by sampling [ Shapiro, Dentcheva & Ruszczynski 2009 ]

48 NON - LINEAR DESIGN Solution through sensitivity calculation and therefore The sensitivities

49 NON - LINEAR DESIGN The reduced gradient Can use steepest descent/LBFGS/Truncated Gauss-Newton to solve the problem

50 CASE STUDY II NON-LINEAR EXPERIMENTAL DESIGN

51 Governing equations Following Finite Element discretization Given model and design settings Find data, n < k Design: find optimal source-receiver configuration IMPEDANCE TOMOGRAPHY – OBSERVATION MODEL

52 IMPEDANCE TOMOGRAPHY – DESIGNS COMPARISON True modelNaive designOptimized design Horesh, Haber & Tenorio 2011

53 MAGNETO - TULLERICS TOMOGRAPHY – OBSERVATION MODEL Governing equations Following Finite Volume discretization Given: model and design settings (frequency ) Find: data, Design: find an optimal set of frequencies

54 MAGNETO - TELLURICS TOMOGRAPHY – DESIGNS COMPARISON Test model Naive design Optimized non-linear designOptimal linearized design Haber, Horesh & Tenorio 2008, 2010

55 THE PARETO CURVE – A DECISION MAKING TOOL Haber, Horesh & Tenorio 2010 Risk To drill or not to drill ? Shakespeare

56 CASE STUDY III REGULARIZATION DICTIONARY DESIGN

57 OPTIMAL OVER-COMPLETE DICTIONARY DESIGN Use popular L 1 to get a sparse solution - but what dictionary should be used ? Accounts for the model space, observation operator and noise characteristics of the problem Requires more sophisticated algorithms to compute Non-smooth optimization framework  Modified L-BFGS [ Overton 2003 ] Horesh & Haber 2009

58 OPTIMAL DICTIONARY DESIGN - NUMERICAL RESULTS Horesh & Haber 2009

59 OPTIMAL DICTIONARY DESIGN - ASSESSMENT WITH NOISE Horesh & Haber 2009

60 OPTIMAL DICTIONARY DESIGN GEOPHYSICAL ASSESSMENT Horesh & Haber 2009

61 CASE STUDY IV REGULARIZATION MATRIX DESIGN

62 OPTIMAL BAYESIAN DESIGN PROBLEM DEFINITION Sparse recovery with a basis (complete, invertible) D Can be equally posed in an analysis form How should we design D ? The posterior involve multivariate Laplace prior distribution

63 OPTIMAL BAYESIAN DESIGN THE OPTIMIZATION PROBLEM Suppose that comprises i.i.d. -Laplace distribution training models The likelihood Estimate D by minimizing the non-linear, non-convex log likelihood Huang, Haber & Horesh (2012)

64 OPTIMAL BAYESIAN DESIGN THE OPTIMIZATION PROBLEM Design preference: sparse D The sub-gradient where Solution with L-BFGS [ Lewis & Overton 2010 ] Huang, Haber & Horesh (2012)

65 OPTIMAL BAYESIAN DESIGN RESULTS - MRI Huang, Haber & Horesh (2012)

66 SUMMARY

67 THE QUOTE CORNER What I see in nature is a grand design that we can comprehend only imperfectly, and that must fill a thinking person with a feeling of humility — A. Einstein Don’t design for everyone. It’s impossible. All you end up doing is designing something that makes everyone unhappy — L. Reichelt It’s art if can’t be explained It’s fashion if no one asks for an explanation It’s design if it doesn’t need explanation — W. Stokkel

68 TAKE HOME MESSAGES Only two (important) elements in the big (inversion) puzzle... Experimental design Regularization design Design in ill-posed inverse problems is an important topic which requires more attention New frontiers in inverse problems and optimization

69 EPILOGUE

70 DESIGN IN INVERSION – OPEN COLLABORATIVE RESEARCH IBM Research MITACS University of British Columbia

71 Andrew Conn Eldad Haber Raya Horesh Jim Nagy Luis Tenorio Andreas Wächter. ACKNOWLEDGMENTS Michele Benzi Jennifer Fohring Hui Haung Ulisses Mello David Nahamoo Alessandro Veneziani DOE NSF. Questions? Thank you

72 PARALLELS AND DISTINCTIONS - CS VS. OED Compressive sensingExperimental design ObjectivesRecover the “best“ model by as few sampling operations as possible Devise Experimental settings that will maximize model information with fewest experiments as possible ConstraintsImpose experimental constraints and preferences ModelSparsely representedArbitrary Sampling operator LocalNon-local Dense (Gaussian, Fourier...)Sparse Linear (linear projections)Non-linear ObservationWell-posedIll-posed


Download ppt "DESIGN IN IMAGING FROM COMPRESSIVE TO COMPREHENSIVE SENSING Lior Horesh Joint work with E. Haber, L. Tenorio, A. Conn, U. Mello, J. Fohring IBM TJ Watson."

Similar presentations


Ads by Google