Download presentation

Presentation is loading. Please wait.

Published byLydia Counter Modified about 1 year ago

1
Multispectral Imaging and Unmixing Jürgen Glatz Chair for Biological Imaging www.cbi.ei.tum.de Munich, 06/06/12

2
Intraoperative Fluorescence Imaging Fluorescence Channel Color Channel

3
Outline Multispectral Imaging Unmixing Methods Exercise: Implementation

4
Multispectral Imaging Multispectral Imaging Unmixing Methods Exercise: Implementation

5
Multispectral Imaging Nature Spatial Resolution Magnification Spatial Resolution Magnification Spectral Resolution Sensitivity Range Spectral Resolution Sensitivity Range Technology Spatial Resolution and Magnification are significantly improved Spectral Resolution has practically not improved since first camera

6
Color Vision Monochrome image of an apple treeColor image of an apple tree Anyone feeling hungry? Color vision helps to distinguish and identify objects against their background (here: fruit and foliage) Color vision provides contrast based on optical properties

7
Color Vision redgreenblue redgreenblue Spectral sensitivity of the human eye longmidshort low light wavelength perception Color receptors (cone cells) with different spectral sensitivity enable trichromatic vision Limited spectral range and poor resolution redgreenblue

8
Limited spectral range Visible Ultraviolet Cleopatra butterflyEvening primrose Human eyes can only see a portion of the light spectrum (ca. 400-750nm) Certain patterns are invisible to the eye

9
Limited spectral resolution Different chemical composition Color vision is insufficient to distinguish between two green objects Differences in the spectra reveal different chemical composition plastic chlorophyll redgreenblueredgreenblue Same color appearance

10
Optical Spectroscopy Spectroscopy analyzes the interaction between optical radiation and a sample (as a function of λ) Provides compositional and structural information Absorbance Fluorescence Transmittance Emission Absorbance Fluorescence Transmittance Emission

11
Directions of optical Methods ImagingSpectroscopy Currently there are two “directions” in optical analysis of an object Camera Spectrometer A B Provides spatial information Provides spectral information Reveals morphological features No information about structure or composition / no spectral analysis Spectrum reveals composition and structure No information about spatial distribution

12
Imaging Spectroscopy Spatial dimension y Spatial dimension x Spectral dimension λ Spatial dimension y Spatial dimension x Spectral dimension λ ImagingSpectroscopy Spatial information Spectral information Imaging Spectroscopy Spectral Cube Spatial and spectral information

13
Spectral Cube λ1λ1 λ2λ2 λ3λ3 λ4λ4 λ5λ5 λ6λ6 λ7λ7 λ8λ8 Acquisition of spatially coregistered images at different wavelengths The maximum number of components that can be distinguished equals the number of spectral bands The accuracy of spectral unmixing increases with the number of bands Chemical compound A Chemical compound B A Bchlorophyllplastic Pseudo-color image representing the distribution of compounds A and B (chlorophyll and plastic) Wavelength

14
Multispectral Imaging Modalities Camera + Filter Wheel Bayer Pattern Cameras + Prism Multispectral Optoacoustic Tomography etc.

15
Let’s find those apples Multispectral imaging alone is only one side of the medal Appropriate data analysis techniques are required to extract information from the measurements

16
Unmixing Methods Multispectral Imaging Unmixing Methods Exercise: Implementation

17
The Unmixing Problem Finding the sources that constitute the measurements For multispectral imaging this means separating image components of different, overlapping spectra Unmixing is a general problem in (multivariate) data analysis Unmixing

18
Multifluorescence Microscopy Disjoint spectra can be separated by bandpass filtering Overlapping emission spectra create crosstalk

19
Autofluorescence Autofluorescence exhibits a broadband spectrum Only mixed observations of the components can be measured Post-processing to unmix them λ I

20
Forward Modeling What constitutes a multispectral measurement at a certain point and wavelength? Principle of superposition: Sum of individual component emission A component‘s emission over different wavelengths λ is denoted by its spectrum, its spatial distribution is still to be defined.

21
Setting up a simple forward problem (1) Two fluorochromes on a homogeneous background Note: We define images as row vectors of length n All components are merged in the (n x k) source matrix O n: Number of image pixels k: Number of spectral components

22
Setting up a simple forward problem (2) Defining the emission spectra for all components at the measurement points Combining them into the (k x m) spectral matrix k: Number of spectral components m: Number of multispectral measurements m ≥ k Wavelength [nm] Relative Absorption [%]

23
Setting up a simple forward problem (3) Two fluorochromes on a homogeneous background Heavily overlapping spectra 25 equidistant measurements under ideal conditions Wavelength [nm] Relative Absorption [%]

24
Mathematical Formulation Multispectral measurement matrix (n x m) Original component matrix (n x k) Spectral mixing matrix (k x m) (+N) Noise, artefacts, etc. (n x m)

25
Multispectral Dataset

26
Mathematical Formulation 10000x2510000x33x25 =

27
Linear Regression: Spectral Fitting Reconstructing O System generally overdetermined: No direct inverse S -1 Generalized inverse: Moore-Penrose Pseudoinvere S + Spectral Fitting: Finding the components that best explain the measurements given the spectra Minimizing the error:

28
Spectral Fitting Orthogonality principle: optimal estimation (in a least squares sense) is orthogonal to observation space

29
Spectral Fitting

30
Given full spectral information (i.e. about all source components) the data can be unmixed Spectral Fitting

31
Blood oxygenation in tumors

32
Multifluorescence Imaging RGB imageFITC TRITCCy3.5Food AutofluorescenceComposite Nude mice with two different species of autofluorescence and three subcutaneous fluorophore signals: FITC, TRITC and Cy3.5. (Totally 5 signals)

33
Spectral Fitting Fast, easy and computationally stable Known order and number of unmixed components Quantitative Requires complete spectral information Crucially depends on accuracy of spectra (systematic errors) Suitable for detection and localization of known compositions

34
? ? Still no apples…

35
Principal Component Analysis Blind source separation (BSS) technique Requires no a priori spectral information Estimates both O and S from M Assumption: Sources are uncorrelated, while mixed measurements are not

36
Principal Component Analysis Unmixing by decorrelation: Orthogonal linear transformation Transforms the data into a space spanned by the orthogonal PCs Maximum variance along first PC, maximum remaining variance along second PC, etc.

37
Unmixing multispectral data with PCA 25 multispectral measurements are correlated Their entire variance can (ideally) be expressed by only 3 PCs Dimension reduction Those 3 PCs are the unmixed sources Note that matrix orientations may vary between different implementations

38
Computing PCA Subtract mean from multispectral observations Covariance Matrix: Diagonalizing C M : Eigenvalue Decomposition Eigenvectors of C M are the principal components, roots of the eigenvalues are the singular values Projecting M onto the PCs: Method 1 (preferred for computational reasons)

39
Computing PCA with the SVD Method 2 (not suitable for implementation) Subtract mean from multispectral observations Singular Value Decomposition: M = UΣV T U is a (m x m) matrix of orthonormal (uncorrelated!) vectors Projecting M onto those decorrelates the measurements Singular values in Σ denote how much variance is explained by the respective PC

40
PCA does more than just unmix U is a (non-quantitative) approximation of the PCs spectra These can be used to verify a components identity Σ is the singular value matrix Relatively small singular values indicate irrelevant components Multispectral data space Original data space PCA UTUT Mixing S (U T ) -1 = U ≈ S

41
PCA Spectra

42

43
Principal Component Analysis (PCA) Needs no a priori spectral information Also reconstructs spectral properties Significance measurement through singular values Unknown order and number of components Generally not quantitative Crucially depends on uncorrelatedness of the sources Suitable for many compounds and identification of unknown components

44
Advanced Blind Source Separation Independent Component Analysis (ICA): assumes statistically independent source components, which is a stronger condition than PCA’s orthogonality Non-negative Matrix Factorization (NNMF): constraint that all elements must be positive Commonly computed by iterative optimization of cost functions, gradient descent, etc.

45
Independent Component Analysis Assumes and requires independent sources: Independence is stronger than uncorrelatedness

46
Independent Component Analysis Central limit theorem: Sum of non-gaussian variables is more gaussian than the individual variables Kurtosis measures non-gaussianity: Maximize kurtosis to find IC Reconstruction:

47
Practical Considerations Noise Artifacts (from reconstruction, reflections, measurement,…) Systematic errors (spectra, laser tuning, illumination,…) Unknown and unwanted components

48
Exercise: Implementation Multispectral Imaging Unmixing Methods Exercise: Implementation

49
Forward Problem / Mixing Define at least 3 non-constant images representing the original components Plot them and store them in the matrix O Define an emission spectrum for every component at an appropriate number of measurment points Plot them and store them in the matrix S Calculate the measurement matrix as M = OS (and save everything)

50
Forward Problem / Mixing Wavelength [nm] Relative Absorption [%] OS

51
Forward Problem / Mixing Change matrices into vectors: y=reshape(X,…) or y=X(:) Plot image from a matrix: imagesc(X) or imshow(X) Useful MatLab functions

52
Spectral Fitting Create an m-file and write a function that Has M and S as input variables Calculates the pseudoinverse S + Returns the unmixing R pinv Test it on your data

53
Spectral Fitting Functions: function [out] = name([input]) Regular matrix inverse: y = inv(x) Useful MatLab functions

54
Principal Component Analysis Create an m-file and write a function that Has M as an input variable Subtracts the mean from the measurements in M Computes the covariance matrix C M Performs an eigenvalue decomposition on C M Sorts the eigenvalues (and corresponding vectors) by size Projects M onto the eigenvectors Returns the projected unmixing, the principal components and their loadings

55
Principal Component Analysis Mean: y = mean(x) Eigenvalue Decomposition: [e_vec e_val] = eig(X) Useful MatLab functions

56
Testing your code Try fitting and PCA on your mixed data Try adding different types and amounts of noise to M (e.g. using imnoise) Simulate systematic errors in your spectra (noise, changing values, offset,…)

57
Independent Component Analysis (voluntary) You can download the FastICA MatLab code from http://research.ics.tkk.fi/ica/fastica/ http://research.ics.tkk.fi/ica/fastica/ Type doc fastica for function description Use the fastica function to unmix your simulated data Compare the result to PCA. What are advantages and disadvantages of ICA?

58
Recommended Reading Shlens, J. – A Tutorial on Principal Component Analysis http://www.cfm.brown.edu/people/gk/APMA2821F/PCA-Tutorial- Intuition_jp.pdf Garini, Y., Young, I.T. and McNamara, G. – Spectral Imaging: Principles and Applications; Cytometry Part A 69A: p.735-747 (2006) http://dx.doi.org/10.1002/cyto.a.20311 http://dx.doi.org/10.1002/cyto.a.20311 Stone, J.V. – A brief Introduction to ICA; Encyclopedia of Statistics in Behavioral Science, Vol. 2, p. 907-912 http://jim- stone.staff.shef.ac.uk/papers/ica_encyc_jvs4everrit2005.pdf

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google