Presentation is loading. Please wait.

Presentation is loading. Please wait.

Multispectral Imaging and Unmixing Jürgen Glatz Chair for Biological Imaging Munich, 06/06/12.

Similar presentations

Presentation on theme: "Multispectral Imaging and Unmixing Jürgen Glatz Chair for Biological Imaging Munich, 06/06/12."— Presentation transcript:

1 Multispectral Imaging and Unmixing Jürgen Glatz Chair for Biological Imaging Munich, 06/06/12

2 Intraoperative Fluorescence Imaging Fluorescence Channel Color Channel

3 Outline  Multispectral Imaging  Unmixing Methods  Exercise: Implementation

4 Multispectral Imaging  Multispectral Imaging  Unmixing Methods  Exercise: Implementation

5 Multispectral Imaging Nature Spatial Resolution Magnification Spatial Resolution Magnification Spectral Resolution Sensitivity Range Spectral Resolution Sensitivity Range Technology Spatial Resolution and Magnification are significantly improved Spectral Resolution has practically not improved since first camera

6 Color Vision Monochrome image of an apple treeColor image of an apple tree Anyone feeling hungry? Color vision helps to distinguish and identify objects against their background (here: fruit and foliage) Color vision provides contrast based on optical properties

7 Color Vision redgreenblue redgreenblue Spectral sensitivity of the human eye longmidshort low light wavelength perception Color receptors (cone cells) with different spectral sensitivity enable trichromatic vision Limited spectral range and poor resolution redgreenblue

8 Limited spectral range Visible Ultraviolet Cleopatra butterflyEvening primrose Human eyes can only see a portion of the light spectrum (ca. 400-750nm) Certain patterns are invisible to the eye

9 Limited spectral resolution Different chemical composition Color vision is insufficient to distinguish between two green objects Differences in the spectra reveal different chemical composition plastic chlorophyll redgreenblueredgreenblue Same color appearance

10 Optical Spectroscopy Spectroscopy analyzes the interaction between optical radiation and a sample (as a function of λ) Provides compositional and structural information Absorbance Fluorescence Transmittance Emission Absorbance Fluorescence Transmittance Emission

11 Directions of optical Methods ImagingSpectroscopy Currently there are two “directions” in optical analysis of an object Camera Spectrometer A B Provides spatial information Provides spectral information Reveals morphological features No information about structure or composition / no spectral analysis Spectrum reveals composition and structure No information about spatial distribution   

12 Imaging Spectroscopy Spatial dimension y Spatial dimension x Spectral dimension λ Spatial dimension y Spatial dimension x Spectral dimension λ ImagingSpectroscopy Spatial information Spectral information Imaging Spectroscopy Spectral Cube Spatial and spectral information

13 Spectral Cube λ1λ1 λ2λ2 λ3λ3 λ4λ4 λ5λ5 λ6λ6 λ7λ7 λ8λ8 Acquisition of spatially coregistered images at different wavelengths The maximum number of components that can be distinguished equals the number of spectral bands The accuracy of spectral unmixing increases with the number of bands Chemical compound A Chemical compound B A Bchlorophyllplastic Pseudo-color image representing the distribution of compounds A and B (chlorophyll and plastic) Wavelength

14 Multispectral Imaging Modalities Camera + Filter Wheel Bayer Pattern Cameras + Prism Multispectral Optoacoustic Tomography etc.

15 Let’s find those apples  Multispectral imaging alone is only one side of the medal  Appropriate data analysis techniques are required to extract information from the measurements

16 Unmixing Methods  Multispectral Imaging  Unmixing Methods  Exercise: Implementation

17 The Unmixing Problem  Finding the sources that constitute the measurements  For multispectral imaging this means separating image components of different, overlapping spectra  Unmixing is a general problem in (multivariate) data analysis Unmixing

18 Multifluorescence Microscopy  Disjoint spectra can be separated by bandpass filtering  Overlapping emission spectra create crosstalk

19 Autofluorescence  Autofluorescence exhibits a broadband spectrum  Only mixed observations of the components can be measured  Post-processing to unmix them λ I

20 Forward Modeling What constitutes a multispectral measurement at a certain point and wavelength? Principle of superposition: Sum of individual component emission A component‘s emission over different wavelengths λ is denoted by its spectrum, its spatial distribution is still to be defined.

21 Setting up a simple forward problem (1)  Two fluorochromes on a homogeneous background  Note: We define images as row vectors of length n  All components are merged in the (n x k) source matrix O n: Number of image pixels k: Number of spectral components

22 Setting up a simple forward problem (2)  Defining the emission spectra for all components at the measurement points  Combining them into the (k x m) spectral matrix k: Number of spectral components m: Number of multispectral measurements m ≥ k Wavelength [nm] Relative Absorption [%]

23 Setting up a simple forward problem (3) Two fluorochromes on a homogeneous background Heavily overlapping spectra 25 equidistant measurements under ideal conditions Wavelength [nm] Relative Absorption [%]

24 Mathematical Formulation Multispectral measurement matrix (n x m) Original component matrix (n x k) Spectral mixing matrix (k x m) (+N) Noise, artefacts, etc. (n x m)

25 Multispectral Dataset

26 Mathematical Formulation 10000x2510000x33x25 =

27 Linear Regression: Spectral Fitting  Reconstructing O System generally overdetermined: No direct inverse S -1 Generalized inverse: Moore-Penrose Pseudoinvere S + Spectral Fitting: Finding the components that best explain the measurements given the spectra Minimizing the error:

28 Spectral Fitting  Orthogonality principle: optimal estimation (in a least squares sense) is orthogonal to observation space 

29 Spectral Fitting

30 Given full spectral information (i.e. about all source components) the data can be unmixed Spectral Fitting

31 Blood oxygenation in tumors

32 Multifluorescence Imaging RGB imageFITC TRITCCy3.5Food AutofluorescenceComposite Nude mice with two different species of autofluorescence and three subcutaneous fluorophore signals: FITC, TRITC and Cy3.5. (Totally 5 signals)

33 Spectral Fitting Fast, easy and computationally stable Known order and number of unmixed components Quantitative Requires complete spectral information Crucially depends on accuracy of spectra (systematic errors)     Suitable for detection and localization of known compositions 

34 ? ? Still no apples…

35 Principal Component Analysis Blind source separation (BSS) technique Requires no a priori spectral information Estimates both O and S from M Assumption: Sources are uncorrelated, while mixed measurements are not

36 Principal Component Analysis Unmixing by decorrelation: Orthogonal linear transformation Transforms the data into a space spanned by the orthogonal PCs Maximum variance along first PC, maximum remaining variance along second PC, etc.

37 Unmixing multispectral data with PCA 25 multispectral measurements are correlated Their entire variance can (ideally) be expressed by only 3 PCs  Dimension reduction Those 3 PCs are the unmixed sources Note that matrix orientations may vary between different implementations

38 Computing PCA Subtract mean from multispectral observations Covariance Matrix: Diagonalizing C M : Eigenvalue Decomposition Eigenvectors of C M are the principal components, roots of the eigenvalues are the singular values Projecting M onto the PCs: Method 1 (preferred for computational reasons)

39 Computing PCA with the SVD Method 2 (not suitable for implementation) Subtract mean from multispectral observations Singular Value Decomposition: M = UΣV T U is a (m x m) matrix of orthonormal (uncorrelated!) vectors Projecting M onto those decorrelates the measurements Singular values in Σ denote how much variance is explained by the respective PC

40 PCA does more than just unmix U is a (non-quantitative) approximation of the PCs spectra These can be used to verify a components identity Σ is the singular value matrix Relatively small singular values indicate irrelevant components Multispectral data space Original data space PCA UTUT Mixing S (U T ) -1 = U ≈ S

41 PCA Spectra


43 Principal Component Analysis (PCA) Needs no a priori spectral information Also reconstructs spectral properties Significance measurement through singular values Unknown order and number of components Generally not quantitative Crucially depends on uncorrelatedness of the sources     Suitable for many compounds and identification of unknown components  

44 Advanced Blind Source Separation  Independent Component Analysis (ICA): assumes statistically independent source components, which is a stronger condition than PCA’s orthogonality  Non-negative Matrix Factorization (NNMF): constraint that all elements must be positive  Commonly computed by iterative optimization of cost functions, gradient descent, etc.

45 Independent Component Analysis Assumes and requires independent sources: Independence is stronger than uncorrelatedness

46 Independent Component Analysis Central limit theorem: Sum of non-gaussian variables is more gaussian than the individual variables Kurtosis measures non-gaussianity: Maximize kurtosis to find IC Reconstruction:

47 Practical Considerations Noise Artifacts (from reconstruction, reflections, measurement,…) Systematic errors (spectra, laser tuning, illumination,…) Unknown and unwanted components

48 Exercise: Implementation  Multispectral Imaging  Unmixing Methods  Exercise: Implementation

49 Forward Problem / Mixing Define at least 3 non-constant images representing the original components Plot them and store them in the matrix O Define an emission spectrum for every component at an appropriate number of measurment points Plot them and store them in the matrix S Calculate the measurement matrix as M = OS (and save everything)

50 Forward Problem / Mixing Wavelength [nm] Relative Absorption [%] OS

51 Forward Problem / Mixing Change matrices into vectors: y=reshape(X,…) or y=X(:) Plot image from a matrix: imagesc(X) or imshow(X) Useful MatLab functions

52 Spectral Fitting Create an m-file and write a function that Has M and S as input variables Calculates the pseudoinverse S + Returns the unmixing R pinv Test it on your data

53 Spectral Fitting Functions: function [out] = name([input]) Regular matrix inverse: y = inv(x) Useful MatLab functions

54 Principal Component Analysis Create an m-file and write a function that Has M as an input variable Subtracts the mean from the measurements in M Computes the covariance matrix C M Performs an eigenvalue decomposition on C M Sorts the eigenvalues (and corresponding vectors) by size Projects M onto the eigenvectors Returns the projected unmixing, the principal components and their loadings

55 Principal Component Analysis Mean: y = mean(x) Eigenvalue Decomposition: [e_vec e_val] = eig(X) Useful MatLab functions

56 Testing your code Try fitting and PCA on your mixed data Try adding different types and amounts of noise to M (e.g. using imnoise) Simulate systematic errors in your spectra (noise, changing values, offset,…)

57 Independent Component Analysis (voluntary)  You can download the FastICA MatLab code from  Type doc fastica for function description  Use the fastica function to unmix your simulated data  Compare the result to PCA. What are advantages and disadvantages of ICA?

58 Recommended Reading Shlens, J. – A Tutorial on Principal Component Analysis Intuition_jp.pdf Garini, Y., Young, I.T. and McNamara, G. – Spectral Imaging: Principles and Applications; Cytometry Part A 69A: p.735-747 (2006) Stone, J.V. – A brief Introduction to ICA; Encyclopedia of Statistics in Behavioral Science, Vol. 2, p. 907-912 http://jim-

Download ppt "Multispectral Imaging and Unmixing Jürgen Glatz Chair for Biological Imaging Munich, 06/06/12."

Similar presentations

Ads by Google