Download presentation

Presentation is loading. Please wait.

Published byTabitha Sagraves Modified about 1 year ago

1
DESIGN IN IMAGING FROM COMPRESSIVE TO COMPREHENSIVE SENSING Lior Horesh Joint work with E. Haber, L. Tenorio, A. Conn, U. Mello, J. Fohring IBM TJ Watson Research Center SIAM Imaging Science 2012, Philadelphia, PA - May 2012

2
INTRODUCTION

3
Aim: infer model Given Experimental design y Measurements Observation model Ill - posedness Naïve inversion... Fails... Need to fill in the missing information EXPOSITION - ILL - POSED INVERSE PROBLEMS

4
PROBLEM CHARACTERISTICS Indirect data (e.g. observation operator non-linear, based on PDEs) Noisy Incomplete Ill-posed Large-scale

5
HOW TO IMPROVE MODEL RECOVERY ? How can we... Improve observation model ? Incorporate more meaningful a-priori information ? Extract more information in the measurement procedure ? Provide more efficient optimization schemes ? Define appropriate distance measures / noise model ?

6
Forward problem (simulation, description) Given: model m & observation model Simulate: data d Inverse problem (estimation, prediction) Given: data d & observation model Infer: model m (and uncertainties) Design (prescription) Given: inversion scheme & observation model Find: the ‘best’ experimental settings y, regularization S, … EVOLUTION OF INVERSION FROM SIMULATION TO DESIGN

7
WHY DESIGN ?

8
WHICH EXPERIMENTAL DESIGN IS BEST ? Design 1Design 3Design 2

9
Where light sources and optodes should be placed ? In what sequence they should be activated ? What laser source intensities should be used ? DESIGN QUESTIONS – DIFFUSE OPTICAL TOMOGRAPHY Arridge 1999

10
What frequencies should be used ? What trajectory of acquisition should be considered ? DESIGN QUESTIONS – ELECTROMAGNETIC INVERSION Newman 1996, Haber & Ascher 2000

11
DESIGN QUESTIONS – SEISMIC INVERSION Clerbout 2000 How simultaneous sources can be used ?

12
DESIGN QUESTIONS – ULTRASOUND IMAGING What levels of ultrasonic waves should be used ? Ludwig 1949

13
How many projections should we collect and at what angles ? How accurate should the projection data be ? DESIGN QUESTIONS – LIMITED ANGLE TOMOGRAPHY

14
WHAT CAN WE DESIGN ?

15
DESIGN EXPERIMENTAL LAYOUT Stonehenge 2500 B.C.

16
DESIGN EXPERIMENTAL PROCESS Galileo Galilei

17
RESPECT EXPERIMENTAL CONSTRAINTS… French nuclear test, Mururoa, 1970

18
HOW TO DESIGN ?

19
THE PHILOSOPHY The goal in designing an experiment is to maximize the expected utility that is chosen to reflect purposes of the experiment The utility is usually defined by measuring the accuracy and fidelity of the information provided by the experiments

20
Previous work Well-posed problems - well established [ Fedorov 1997, Pukelsheim 2006 ] Ill-posed problems - under-researched [ Curtis 1999, Bardow 2008 ] Many practical problems in engineering and sciences are ill-posed What makes ill-posed problems so special ? ILL VS. WELL - POSED OPTIMAL EXPERIMENTAL DESIGN

21
OPTIMALITY CRITERIA IN WELL-POSED PROBLEMS For linear inversion, employ Tikhonov regularized least squares solution Bias - variance decomposition For over-determined problems A-optimal design problem

22
OPTIMALITY CRITERIA IN WELL-POSED PROBLEMS Optimality criteria of the information matrix A-optimal design average variance D-optimality uncertainty ellipsoid E-optimality minimax Almost a complete alphabet… Stone, DeGroot and Bernardo

23
THE PROBLEMS... Ill-posedness controlling variance alone reduces the error mildly [ Johansen 1996 ] Non-linearity bias-variance decomposition is impossible What strategy can be used ? Proposition 1 - Common practice so far Trial and Error…

24
EXPERIMENTAL DESIGN BY TRIAL AND ERROR Pick a model Run observation model of different experimental designs, and get data Invert and compare recovered models Choose the experimental design that provides the best model recovery

25
THE PROBLEMS... Ill-posedness controlling variance alone reduces the error mildly [ Johansen 1996 ] Non-linearity bias-variance decomposition is impossible What other strategy can be used ? Proposition 2 - Minimize bias and variance altogether How to define the optimality criterion ?

26
OPTIMALITY CRITERIA FOR DESIGN Loss Mean Square Error Bayes risk Depends on the noise Depends on unknown model Computationally infeasible

27
OPTIMALITY CRITERIA FOR DESIGN Bayes empirical risk Assume a set of authentic model examples is available Discrepancy between training and recovered models [ Vapnik 1998 ] How can y and S be regularized ? Regularized Bayesian empirical risk

28
OTHER DESIGNERS / KEY PLAYERS This is one doctrine Other interesting choices were developed by Y. Marzouk et. al, MIT (2011, 2012) A. Curtis et. al, University of Edinburgh (1999, 2010) D. Coles & M. Prange, Schlumberger (2008, 2012) S. Körkel et. al, Heidelberg University (2011) A. Bardow, RWTH Aachen University (2008, 2009)

29
DIFFERENTIABLE OBSERVATION SPARSITY CONTROLLED DESIGN Assume: fixed number of observations Design preference: small number of sources / receivers is activated The observation model Regularized risk Horesh, Haber & Tenorio 2011

30
DIFFERENTIABLE OBSERVATION SPARSITY CONTROLLED DESIGN Total number of observations may be large Derivatives of the forward operator w.r.t. y Effective when activation of each source and receiver is expensive Horesh, Haber & Tenorio 2011 Difficult…

31
WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN Assume: a predefined set of candidate experimental setups is given Design preference: small number of observations Haber, Horesh & Tenorio 2010

32
WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN Let be discretization of the space [ Pukelsheim 1994 ] Let The (candidates) observation operator is weighted w inverse of standard deviation - reasonable standard deviation - conduct the experiment - infinite standard deviation - do not conduct the experiment Haber, Horesh & Tenorio 2010

33
WEIGHTS FORMULATION SPARSITY CONTROLLED DESIGN Solution - more observations give better recovery Desired solution many w‘s are 0 Add penalty to promote sparsity in observations selection Less degrees of freedom No explicit access to the observation operator needed

34
THE OPTIMIZATION PROBLEMS Leads to a stochastic bi-level optimization problem Direct formulation Weights formulation Haber, Horesh & Tenorio 2010, 2011

35
THE OPTIMIZATION PROBLEM(S) In general (multi-level) optimization problem Difficult to solve (although not impossible) [ Alexaderov & Denis 1994 ] To simplify make assumptions on F - linear, nonlinear S - quadratic, matrix form y - discrete, continuous Important - sufficient improvement can be “good enough”

36
HOW MANY SAMPLES ARE NEEDED ? The optimist - choose a single m and design for it The pessimist - assume that m is the worst it can be The realist - choose an ensemble of m’s and average A Bayesian estimate of the frequentist risk [ Rubin 1984 ]

37
HOW MANY SAMPLES ARE NEEDED ? What type of character are you ? The Optimist The Realist The Pessimist

38
OPTIMAL DESIGN IN PRACTICE

39
LINEAR DESIGN

40
Assume Then

41
LINEAR DESIGN Assume m has second moment Then Best linear design

42
CASE STUDY I LINEAR EXPERIMENTAL DESIGN

43
THE BEST (LINEAR) EXPERIMENT Considering discretization of the design space We obtain

44
BOREHOLE RAY TOMOGRAPHY - OBSERVATION MODEL Use 32 2 × 3 = 3072 rays Goal: choose 500 optimal observations

45
BOREHOLE RAY TOMOGRAPHY EXPERIMENTAL DESIGN ASSESSMENT Assess design performance upon unseen models (cross validation) Test modelOptimal designNon-optimal design

46
NON - LINEAR DESIGN

47
Use Stochastic Optimization - approximate by sampling [ Shapiro, Dentcheva & Ruszczynski 2009 ]

48
NON - LINEAR DESIGN Solution through sensitivity calculation and therefore The sensitivities

49
NON - LINEAR DESIGN The reduced gradient Can use steepest descent/LBFGS/Truncated Gauss-Newton to solve the problem

50
CASE STUDY II NON-LINEAR EXPERIMENTAL DESIGN

51
Governing equations Following Finite Element discretization Given model and design settings Find data, n < k Design: find optimal source-receiver configuration IMPEDANCE TOMOGRAPHY – OBSERVATION MODEL

52
IMPEDANCE TOMOGRAPHY – DESIGNS COMPARISON True modelNaive designOptimized design Horesh, Haber & Tenorio 2011

53
MAGNETO - TULLERICS TOMOGRAPHY – OBSERVATION MODEL Governing equations Following Finite Volume discretization Given: model and design settings (frequency ) Find: data, Design: find an optimal set of frequencies

54
MAGNETO - TELLURICS TOMOGRAPHY – DESIGNS COMPARISON Test model Naive design Optimized non-linear designOptimal linearized design Haber, Horesh & Tenorio 2008, 2010

55
THE PARETO CURVE – A DECISION MAKING TOOL Haber, Horesh & Tenorio 2010 Risk To drill or not to drill ? Shakespeare

56
CASE STUDY III REGULARIZATION DICTIONARY DESIGN

57
OPTIMAL OVER-COMPLETE DICTIONARY DESIGN Use popular L 1 to get a sparse solution - but what dictionary should be used ? Accounts for the model space, observation operator and noise characteristics of the problem Requires more sophisticated algorithms to compute Non-smooth optimization framework Modified L-BFGS [ Overton 2003 ] Horesh & Haber 2009

58
OPTIMAL DICTIONARY DESIGN - NUMERICAL RESULTS Horesh & Haber 2009

59
OPTIMAL DICTIONARY DESIGN - ASSESSMENT WITH NOISE Horesh & Haber 2009

60
OPTIMAL DICTIONARY DESIGN GEOPHYSICAL ASSESSMENT Horesh & Haber 2009

61
CASE STUDY IV REGULARIZATION MATRIX DESIGN

62
OPTIMAL BAYESIAN DESIGN PROBLEM DEFINITION Sparse recovery with a basis (complete, invertible) D Can be equally posed in an analysis form How should we design D ? The posterior involve multivariate Laplace prior distribution

63
OPTIMAL BAYESIAN DESIGN THE OPTIMIZATION PROBLEM Suppose that comprises i.i.d. -Laplace distribution training models The likelihood Estimate D by minimizing the non-linear, non-convex log likelihood Huang, Haber & Horesh (2012)

64
OPTIMAL BAYESIAN DESIGN THE OPTIMIZATION PROBLEM Design preference: sparse D The sub-gradient where Solution with L-BFGS [ Lewis & Overton 2010 ] Huang, Haber & Horesh (2012)

65
OPTIMAL BAYESIAN DESIGN RESULTS - MRI Huang, Haber & Horesh (2012)

66
SUMMARY

67
THE QUOTE CORNER What I see in nature is a grand design that we can comprehend only imperfectly, and that must fill a thinking person with a feeling of humility — A. Einstein Don’t design for everyone. It’s impossible. All you end up doing is designing something that makes everyone unhappy — L. Reichelt It’s art if can’t be explained It’s fashion if no one asks for an explanation It’s design if it doesn’t need explanation — W. Stokkel

68
TAKE HOME MESSAGES Only two (important) elements in the big (inversion) puzzle... Experimental design Regularization design Design in ill-posed inverse problems is an important topic which requires more attention New frontiers in inverse problems and optimization

69
EPILOGUE

70
DESIGN IN INVERSION – OPEN COLLABORATIVE RESEARCH IBM Research MITACS University of British Columbia

71
Andrew Conn Eldad Haber Raya Horesh Jim Nagy Luis Tenorio Andreas Wächter. ACKNOWLEDGMENTS Michele Benzi Jennifer Fohring Hui Haung Ulisses Mello David Nahamoo Alessandro Veneziani DOE NSF. Questions? Thank you

72
PARALLELS AND DISTINCTIONS - CS VS. OED Compressive sensingExperimental design ObjectivesRecover the “best“ model by as few sampling operations as possible Devise Experimental settings that will maximize model information with fewest experiments as possible ConstraintsImpose experimental constraints and preferences ModelSparsely representedArbitrary Sampling operator LocalNon-local Dense (Gaussian, Fourier...)Sparse Linear (linear projections)Non-linear ObservationWell-posedIll-posed

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google