1. Multiplexed Fluorescence Unmixing Marina Alterman, Yoav Schechner Aryeh Weiss Technion, Israel Bar-Ilan, Israel

2. 2 Natural Linear Mixing Raskar et al. 2006. ImageJ image sample collection. c ci i c i

3. 3 Natural Linear Mixing ? ImageJ image sample collection. c Raskar et al. 2006. ci i How do you measure i ? c i

4. Single Source Excitation Multiplexed Excitation 4 demultiplex i1i1 i2i2 i3i3 1 a3a3 2 3 Beam combiner a1a1 1 2 a2a2 3 3 1 2 a = 0 1 1 i a 1 1 0 i a 1 0 1 i 1 2 3 1 2 3

5. Why Multiplexing? + noise SNR Trivial Measurements SNR Multiplexed Measurements Same acquisition time 5 Intensity vector i

6. Multiplexing - Look closer 6 Xc i i – single source intensities η - noise estimation acquisition Minimum  W=? Estimate c not i

7. 7 a i ˆ WiWi a i ˆ c ˆ WcWc Common ApproachThis Work c ˆ Concentrations Single source intensities Acquired multiplexed intensities efficient acquisition Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing N dyes =3 N sources =7 size(i)=7 Wi≠WcWi≠Wc Multiplexing: a=Wi, Mixing: i=Xc

8. Fluorescence 8 http://www.microscopyu.com/galleries/fluorescence, http://www.microscopy.fsu.edu/primer/techniques/fluorescence/fluorogallery.html Cell structure and processes Corn Grain Flea Intestine Tissue Horse Dermal Fibroblast Cells Fluorescent Specimen

9. 9 Linear Mixing Molecules per pixel More molecules per pixel Brighter pixel c i i c i = x∙c Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing

10. 10 Linear Mixing {c d } i vector of concentrations (spatial distribution) For each pixel: i = x x ∙ ∙ ∙ x 1 2 N dyes cc∙∙∙ccc∙∙∙c 1 2 N dyes Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing

11. 11 Linear Mixing s=1 i vector of intensities Mixing matrix cc∙∙∙ccc∙∙∙c 1 2 N dyes 1 s=2 i 2 i = x x ∙ ∙ ∙ x 1,1 1,2 1,N dyes 1 i = x x ∙ ∙ ∙ x 2,1 2,2 2,N dyes 2 i = x x ∙ ∙ ∙ x s,1 s,2 s,N dyes s ∙∙∙∙∙∙ {c d } vector of concentrations (spatial distribution) For each pixel:

12. 12 Linear Mixing s=1 i vector of intensities Mixing matrix For each pixel: s=2 i 2 1 {c d } vector of concentrations (spatial distribution)

13. Fluorescent Microscope Fluorescent Specimen Dichroic Mirror Emission Filter Intensity image Blue L 2 (λ) 13 300 400 500 600 700 λ λ Excitation Sources Excitation Filter s=1 s=2 s=3 s=4 =5s s : illumination sources e(λ) 300 400 500 600 700 λ α(λ)α(λ)

14. Intensity image Fluorescent Microscope Fluorescent Specimen Dichroic Mirror Emission Filter Green L 2 (λ) 300 400 500 600 700 λ λ Excitation Sources Excitation Filter s=1 s=2 s=3 s=4 =5s s : illumination sources 300 400 500 600 700 λ α(λ)α(λ) e(λ) Cross-talk 14 Unmixing required Intensity image (mixed) Blue

15. Problem Definition 15 Unmix Intensity image (mixed) + noise noise How to multiplex for least noisy unmixing? Fluorescent specimen Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing

16. Sum up the concepts c ia mixing un mixing multiplexing de multiplexing Concentrations Single source Image intensities Acquired multiplexed image intensities X X -1 W W -1 Nature Man made 16 Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing multiplexed unmixing

17. Look closer - again 17 Xc i Estimate c not i i – single source intensities η - noise Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing

18. Multiplexed Unmixing acquisition Minimum Variance in c  W=? For each pixel 18 i c = + a acquired measurements W multiplexing matrix X mixing matrix noise estimation OR Weighted Least Squares WX is not square Other estimators OR

19. Generalizations 19 var(η) = constant i =? c =? var(η) ≠ constant Image intensities concentrations Minimum Var  W=? η - noise Details in the paper Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing

20. Generalized Multiplex Gain 20 What is the SNR gain for unmixing? Only Unmixing Unmixing + Multiplexing VS. Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing

21. Significance of the Model N sources =N measure 34567 1 1.2 1.4 1.6 1.8 2 2.2 21 GAIN c a i ˆ c ˆ WcWc a i ˆ c ˆ WiWi VS. Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing Wi≠WcWi≠Wc

22. Significance of the Model N sources =N measure 34567 1 1.2 1.4 1.6 1.8 2 2.2 22 GAIN c a i ˆ c ˆ WcWc Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing

23. Significance of the Model N sources =N measure 34567 1 1.2 1.4 1.6 1.8 2 2.2 23 GAIN c GAIN < 1 For specific 3 dyes, camera and filter characteristics a i ˆ c ˆ WiWi a i ˆ c ˆ WcWc

24. 24 Natural Linear Mixing ? ImageJ image sample collection. c Raskar et al. 2006. ci i c i

25. = + a X c W Multiplexed Unmixing 25 η i Xc The goal is unmixing Efficient Acquisition Exploit all available sources SNR improvement Generalization of multiplexing theory Alterman, Schechner & Weiss, Multiplexed Fluorescence Unmixing