1. Basics of Deconvolution
Presented by:
Andrew Molnar
Senior Software Engineer, Media Cybernetics

2. What is Deconvolution?
Deconvolution is an image restoration operation that assists in reversing the blurring effects of an optical system.
It is distinct from deblurring (nearest neighbors, unsharp masking, etc.) in that it reassigns, rather than removes, out of focus light.

3. Why Deconvolution? The image is not the sample.
The collected image is distorted by the optics, illumination, noise, as well as the sample itself.
All images have distortions and limitations, no matter how good they look.
Deconvolution reveals structure obscured by the distortions inherent in any imaging process.

4. Object convolved with the PSF, added to Noise, equals the Image
Image Formation
Object
PSF
Noise
Image  + =
Object convolved with the PSF, added to Noise, equals the Image
Each object in the optical path causes some blur.
The blur of a single point of light is called the point-spread function (PSF).
The PSF, applied to all points in the image, forms the observed image.
Additionally, noise is introduced through a combination of ambient factors and imaging hardware.

5. Light Diffraction – Airy Disc
Light through an objective converges and interferes at the image plane
Interference pattern is called an Airy pattern
At the center is an Airy disc

6. Resolution and Blur
Structures separated by more than an Airy disc are resolvable.
Any structures separated by less than an Airy disc are blurred together and appear as one structure

7. Deconvolution Overview
Acquire microscope data
Generate/Collect PSF
Deconvolve

8. Acquisition Criteria
Sample interval (Nyquist): >2x deconvolved resolution
XY: Z:
100x 1.4NA 550nm: XY < 0.07 mm, Z < 0.3 mm
Acquire slices several DOF above/below sample
Minimize refractive index mismatch
Best case: immersion RI = sample RI
Do not saturate pixels!
The best results come from the best data
Recommend sampling 2.3 to 2.5x the highest resolution
Z slices < 0.25 are not worthwhile even with the best objectives

9. Deconvolution Overview
Acquire microscope data
Generate/Acquire PSF
Deconvolve

10. Determining the PSF
Just as important (perhaps more so) as choosing the correct deconvolution approach
Measured PSF: Acquire a point-object image to use as a PSF
Theoretical PSF: Generate a model PSF from optical parameters

11. Measured PSF: How to collect
Image a bead with a size ideally smaller than the resolution limit of the microscope
Acquisition must be done under the same conditions as the sample
Pixel calibration mismatches can be corrected
Can be imaged simultaneously with the sample: include a bead on the slide, isolated from the sample

12. Theoretical PSF: What affects the shape?
Scope Modality
Must be a model supported by the software to generate a theoretical PSF
Example: AutoQuant X3 supports:
Widefield Fluorescence
Transmitted-Light Brightfield
Laser Scanning Confocal
Spinning Disk Confocal
Multi-Photon Fluorescence
Unknown modalities can still be deconvolved using a measured PSF!

13. Theoretical PSF: What affects the shape?
Lens parameters
Numerical Aperture
Refractive Index of Lens Immersion
Pixel Calibration
Emission Wavelength
Specimen Parameters
Refraction Index of Specimen Embedding
Specimen Distance from Coverslip
Spherical Aberration XY XY XZ XZ
Widefield
Confocal

14. Spherical Aberration Caused by Solve by Match RI
Changes in refractive index
Thick specimens
Incorrect cover slip thickness
Worse deeper in the sample
Solve by
Match RI
SA correction collar
Use biased PSF and deconvolution

15. SA: PSF changes with depth
SA gets worse deeper into sample
Most deconvolution algorithms assume constant PSF
Calculate/Collect PSF at same depth (z) as objects in the sample.

16. Choosing a PSF
Theoretical: Best for supported modalities when optical parameters are known, or when a good bead image is not available.
Measured: Best when a good bead image is available, or for unsupported modalities (STED, SI, etc.).

17. Deconvolution Overview
Acquire microscope data
Generate/Acquire PSF
Deconvolve

18. Approaches to Deconvolution
Inverse Filter
Constrained Iterative
Statistical Constrained Iterative
Non-blind
Blind

19. Inverse Filter: Solve for Object directly1 ? = 
Object
Object becomes image convolved with inverse of PSF
Weiner, Regularized Least Squares.
Pros
Relatively quick (non-iterative)
Reasonable results when signal to noise is high
Cons
Does not address noise directly
Breaks down with inaccurate PSF

20. Constrained Iterative Deconvolution Algorithms
Optimization approach
Work by improving solution through repeated iterations.
Constraints
Non-negativity
Filter Gaussian Noise
Jansen-Van Cittert, Least Squares.

21. Statistical Constrained Iterative Deconvolution
Optimization is done using statistical criteria (Maximum Likelihood Estimation)
Better noise handling
Poisson noise
Most likely estimate of object given that photon emission is Poisson.
Most (if not all) modern deconvolution packages use statistical constrained iterative methods

22. Statistical Constrained Iterative: Blind and Non-Blind
Adaptive PSF (Blind)
Both image and PSF estimate are changed as part of iterative optimization
Can adapt to imperfections in the microscope
Can adapt to specimen refractive index
Handles spherical aberration well
Fixed PSF (Non-blind)
PSF is fixed; remains unchanged with each deconvolution iteration
Only the image estimate is updated at each iteration
More consistent processing across volumes

23. Iterative Deconvolution
(Fixed PSF)
Object Estimate
PSF
Image Estimate
Image -
Update
Constraints
Error
First Iteration

24. Deconvolution Data Improvements
Increased resolution
Higher contrast, reduced haze
Noise suppression
Resolution increase of ~30% in XY, roughly equivalent to confocal depending on sample structure
Richard Cole, NYS Department of Health, Biggs Laboratory- Wadsworth Center, Albany NY 12201

25. Improvements are Additive
Deconvolve confocal data
Improve resolution
Improve S/N:
Noise without a PSF
will be suppressed

26. Uses - Increase Contrast and Resolution

27. Uses – Measurement/Classification
AA erythrocyte
Fuyuki Tokumasu
National Institute of Allergy and Infectious Diseases
Journal of Cell Science 118 ,

28. Uses - Analysis Raw Image Deconvolved Threshold Dan Mulvihill
Cell Developmental Biology Group
University of Kent

29. Some Side Effects of Deconvolution
Increased dynamic range
12-bit unprocessed image may require 16 or 32 bits to store the result
Caused by reassignment of light to point sources
Image may appear darker overall
Bright spots are much brighter, but are much fewer in number

30. Potential Issues Generally blurred or noisy results
Check noise estimate
Remaining blur, “ringing” (see individual slices)
Try additional or fewer iterations
Examine XZ, YZ slices
If an obvious PSF blur remains, the PSF may be inaccurate

31. Frequently Asked Questions
How do I know my data isn’t “Photoshopped”?
Deconvolving known structures (beads, microtubules, etc) demonstrates that deconvolution accurately restores those structures; within the information limits of the microscope
Is deconvolved data quantitative?
Note: fluorescence imaging does not provide absolute intensities outside single molecule or internal reference (ratio, FRET) imaging, it gives relative intensities
Deconvolution is more quantitative than the raw data: lower noise, less out of focus haze contamination, more accurate spatial information (edges, positions)

32. Validation Publication
Calibration of Wide-Field Deconvolution Microscopy for Quantitative Fluorescence Imaging
J Lee, T Wee, C Brown – Journal of Biomolecular Techniques – ABRF
Raw
Deconvolved
Follows “best practices” for image collection.
Measures multiple sets of bead collections before and after deconvolution.
Intensity analysis determines that “… the deconvolved images clearly indicate that deconvolution using the AutoQuant X3 blind deconvolution algorithm preserves the relative quantitative intensity data.”
Volume analysis finds that “deconvolution decreased the measured volume to 70% of the original image volumes, which is much closer to the expected volume… based on the actual microsphere size.”

33. Questions?