1. Computational Biology, Part 22 Biological Imaging I G. Steven Vanni Meel Velliste Robert F. Murphy Copyright  1998, 2000-2006. All rights reserved.

2. Biological imaging Significant advances in the fields of optics and electronics in the past two decades have greatly increased the utility of imaging for addressing biological questions. Significant advances in the fields of optics and electronics in the past two decades have greatly increased the utility of imaging for addressing biological questions. These advances permit These advances permit  more diverse types of information to be extracted from biological specimens  with greater accuracy  and under more demanding conditions.

3. Imaging relies on generating a detectable signal which can be used as a measure of a property of interest in the specimen. Imaging relies on generating a detectable signal which can be used as a measure of a property of interest in the specimen. This property of interest is the initial signal, but it must be transduced or changed through several forms before it becomes detectable. This property of interest is the initial signal, but it must be transduced or changed through several forms before it becomes detectable. Chemical and molecular biological probes may be targeted within a specimen

4. For example: A protein may be modified so that when it enters a cell and bumps into another protein involved in a specific activity, it fluoresces. The original activity was probably not detectable, but this newly generated fluorescence signal is detectable. For example: A protein may be modified so that when it enters a cell and bumps into another protein involved in a specific activity, it fluoresces. The original activity was probably not detectable, but this newly generated fluorescence signal is detectable. Front end of imaging system and detector Specimen Specimen may be difficult to see except where labeled by probe. Chemical and molecular biological probes may be targeted within a specimen

5. Image Formation and Acquisition Having an understanding of the specimen, the next step is the formation and acquisition of a digital image Having an understanding of the specimen, the next step is the formation and acquisition of a digital image A two dimensional image plane consists of a rectangular grid of points, or pixels A two dimensional image plane consists of a rectangular grid of points, or pixels Grid Specimen Pixel

6. A digital image plane is acquired by recording a digital value proportional to the intensity of light (or other form of energy) impinging on each pixel of a detector A digital image plane is acquired by recording a digital value proportional to the intensity of light (or other form of energy) impinging on each pixel of a detector This intensity usually corresponds to the amount of light emitted by or reflected from a corresponding point on a specimen This intensity usually corresponds to the amount of light emitted by or reflected from a corresponding point on a specimen 0002100 0001000 0378300 0688820 0288840 0048830 Projection of specimen onto dectector grid Image Specimen Image Formation and Acquisition

7. “Pixel” is used interchangeably to mean: One of multiple regions on a detector, each corresponding to the smallest area from which a signal can be distinguished One of multiple regions on a detector, each corresponding to the smallest area from which a signal can be distinguished The numerical value associated with each such region in a digital image The numerical value associated with each such region in a digital image A region on display device, such as a monitor or printer A region on display device, such as a monitor or printer 0002100 0001000 0378300 0688820 0288840 0048830 Dectector gridPixel valuesDisplay device 7-8 4-6 1-3 0 Pixel

8. Display of pixel values A pixel value is just a number in the data set representing a digital image. A pixel value is just a number in the data set representing a digital image. Pixel values may be displayed in different ways, determined by a look up table (LUT). Pixel values may be displayed in different ways, determined by a look up table (LUT). 0002100 0001000 0378300 0688820 0288840 0048830 Pixel valuesHot to cold 7-8 4-6 1-3 0 Arbitrary 7-8 4-6 1-3 0 Binary 1-8 0 LUT

9. Image Formation Biological images may be acquired via a variety of imaging modes or modalities Biological images may be acquired via a variety of imaging modes or modalities Each mode is a combination of an image formation system and a detector Each mode is a combination of an image formation system and a detector

10. Image formation system Sample Image formation system Detector

11. Detector and image types While the examples so far have dealt with light microscope images, we will now back up for a microscope images, we will now back up for a few minutes to consider many different types few minutes to consider many different types of images before concentrating again on light of images before concentrating again on light microscopy. microscopy.

12. Detector and image types In general, images may be classified according to what is being detected: (Visible) light transmission, scattering or emission (Visible) light transmission, scattering or emission  single wavelength, 3 color, or full spectrum Electron transmission or scattering Electron transmission or scattering X-ray transmission X-ray transmission Radioactive particle emission Radioactive particle emission Magnetic field perturbation Magnetic field perturbation Physical displacement from “atomic force” Physical displacement from “atomic force”

13. Comparing types of imaging

14. Light microscopy Three primary types of detectors Three primary types of detectors  human eye  no digital image  CCD or charge coupled device  “work horse” of modern biological imaging  acquires digital image directly  PMT or photomultiplier tube  scans to produce digital image

15. CCD chips Penny A CCD chip is the actual detector within a CCD camera.

16. Light sources in the object Consider a fluorescent specimen made of individual molecules of fluorescent dye. Consider a fluorescent specimen made of individual molecules of fluorescent dye.  Each molecule can emit light.  Each dye molecule may be seen as a vanishingly small emitter.  Such an emitter is called a point source.  The concept of a point source is useful because a point is simple to model, and if we know how a point source is imaged, then we can easily model a complex specimen as a combination of many points and predict how it will be imaged.

17. Fluorescence Microscope Objective Arc Lamp Emission Filter Excitation Diaphragm Ocular Excitation Filter

18. Light sources in the object A specific example might be a microscope slide containing cells stained with fluorescent dye. A specific example might be a microscope slide containing cells stained with fluorescent dye. In an ideal image, a point source would show intensity in only one pixel In an ideal image, a point source would show intensity in only one pixel

19. In reality, the light from each point in the specimen is seen to spread out and affect many pixels in the image. In reality, the light from each point in the specimen is seen to spread out and affect many pixels in the image. This “spreading” is an inescapable result of the optical properties of the image formation system. This “spreading” is an inescapable result of the optical properties of the image formation system. The mathematical description of this spreading or blurring process is called a point-spread function (PSF) The mathematical description of this spreading or blurring process is called a point-spread function (PSF) Point-spread function

20. The point-spread function (PSF) is determined by the optics of the image formation system, including factors such as the refractive index, diameter and magnification of its components The point-spread function (PSF) is determined by the optics of the image formation system, including factors such as the refractive index, diameter and magnification of its components The resulting blurred region in the image can be approximated by a 2D Gaussian distribution The resulting blurred region in the image can be approximated by a 2D Gaussian distribution

21. Realistic Image of a Point Source This graph shows intensity on the z-axis for a PSF defined in the X-Y plane. This graph shows intensity on the z-axis for a PSF defined in the X-Y plane. Later we will consider a PSF defined in three dimensions. Later we will consider a PSF defined in three dimensions.

22. Light sources in the object Thus, when a 2D image is acquired, each point in the specimen will be blurred in all directions and will contribute to the recording in many pixels around that pixel to which it directly corresponds Thus, when a 2D image is acquired, each point in the specimen will be blurred in all directions and will contribute to the recording in many pixels around that pixel to which it directly corresponds

23. Introduction to 3D Microscopy The spreading of light from a point source actually occurs in three dimensions as will be shown. The spreading of light from a point source actually occurs in three dimensions as will be shown. First, however, it is necessary to understand the three dimensional (3D) nature of the object and image as acquired via 3D microscopy. First, however, it is necessary to understand the three dimensional (3D) nature of the object and image as acquired via 3D microscopy.

24. When a microscope is focused on a specimen, the detector records an image from a plane. When a microscope is focused on a specimen, the detector records an image from a plane.  This is the focal plane.  Parts of the specimen in the focal plane are in the best focus. Detector Focalplane 3D Microscopy

25. 3D data is acquired by combining data from several different focal planes into a stack of images. 3D data is acquired by combining data from several different focal planes into a stack of images. This is accomplished by changing the distance between the specimen and the microscope’s objective lens from one image acquisition to the next. This is accomplished by changing the distance between the specimen and the microscope’s objective lens from one image acquisition to the next. Objective Imagestack

26. The next slide shows a real 3D image stack. The next slide shows a real 3D image stack. The specimen is a HeLa cell labeled with a antibody against the cytoskeletal protein tubulin and a secondary antibody conjugated to a fluorescent dye. The specimen is a HeLa cell labeled with a antibody against the cytoskeletal protein tubulin and a secondary antibody conjugated to a fluorescent dye. The images were acquired using a confocal fluorescence microscope. The images were acquired using a confocal fluorescence microscope. The image stack is presented here as a movie with one acquired image plane per movie frame. The image stack is presented here as a movie with one acquired image plane per movie frame. Real 3D image data

27. Microtubules in a human cell Courtesy of Meel Velliste

28. Now, with a better understanding of what makes up a 3D image stack, we can better consider how light from a point source spreads out and is imaged in three dimensions. Now, with a better understanding of what makes up a 3D image stack, we can better consider how light from a point source spreads out and is imaged in three dimensions. On the following slide, intensity is shown by variation in color. On the following slide, intensity is shown by variation in color.  Warm colors indicate greater intensity.  All axes indicate real spatial dimensions as indicated. Real 3D image of a point source

29. yz xx Courtesy of Image & Graphics Inc.: http://www.imagepro.co.kr/ 3D Reconstruction of Point Spread Function (PSF) from 0.2 Micron Bead NOTE: Spreading along the Z-axis is more pronounced. Increasing intensity

30. Image Formation Image formation can be described as: Image formation can be described as:  the convolution of an array describing the original specimen or object  with a function describing the image formation system  to yield an acquired image.

31. A convolution may be written in somewhat simplified mathematical form as follows: A convolution may be written in somewhat simplified mathematical form as follows: The concept of a convolution i(x,y,z) defines the image in its 3D space according to the form of the equation above. i(x,y,z) defines the image in its 3D space according to the form of the equation above. PSF(x-x’,y-y’,z-z’) defines the amount of light from a point source at x’,y’,z’ that will be observed at x,y,z PSF(x-x’,y-y’,z-z’) defines the amount of light from a point source at x’,y’,z’ that will be observed at x,y,z o(x’,y’,z’) describes the specimen or object. o(x’,y’,z’) describes the specimen or object.

32. Image Formation The mathematical view of convolution emphasizes that each point in the sample can contribute to each point in the image The mathematical view of convolution emphasizes that each point in the sample can contribute to each point in the image

33. Widefield Fluorescence Microscopy This type of fluorescence microscope collects light emitted from all points in the specimen (with varying efficiencies depending on position relative to focal plane) The result for specimens that are thick relative to the depth of focus of the objective is a blurred image

34. Confocal Microscopy One way to obtain images that better represent the fluorescence distribution just in the focal plane is to use a confocal microscope One way to obtain images that better represent the fluorescence distribution just in the focal plane is to use a confocal microscope

35. Confocal Microscope Principle Objective Laser Emission Pinhole Excitation Pinhole PMT Emission Filter Excitation Filter

36. http://micro.magnet.fsu.edu/primer/confocal/index.html

39. Benefits of Confocal Microscopy Reduced blurring of the image from light scattering Reduced blurring of the image from light scattering Increased effective resolution Increased effective resolution Improved signal to noise ratio Improved signal to noise ratio Clear examination of thick specimens Clear examination of thick specimens Z-axis scanning Z-axis scanning Depth perception in Z-sectioned images Depth perception in Z-sectioned images Magnification can be adjusted electronically Magnification can be adjusted electronically

40. Drawbacks of Confocal Microscopy Slower acquisition - need to collect one pixel at a time Slower acquisition - need to collect one pixel at a time Increased photodamage (photobleaching) due to longer exposure to exciting light Increased photodamage (photobleaching) due to longer exposure to exciting light

41. Spinning Disk Confocal Microscopy To allow faster acquisition, do confocal imaging “in parallel” To allow faster acquisition, do confocal imaging “in parallel”

42. Image Formats An bit map image normally consists of an 8- bit or 16-bit value for each pixel. An bit map image normally consists of an 8- bit or 16-bit value for each pixel. These values are stored as computer files in various formats. These values are stored as computer files in various formats. Pixel values are normally stored linearly in a file with the values for the first row of pixels followed immediately by the values for the second row (etc.). Pixel values are normally stored linearly in a file with the values for the first row of pixels followed immediately by the values for the second row (etc.).

43. Image Formats At a minimum, an image format contains: At a minimum, an image format contains:  Image size (# of rows and columns)  Number of bits per pixel  Order in which bytes within words are stored  Number of bytes to skip at the beginning of the image (the offset)  The beginning of image files often has a text header that can be skipped if the above values are known.  This header may contain additional descriptive information about the image such as: subject of imagesubject of image name of person and/or application creating the imagename of person and/or application creating the image

44. Common Image File Formats TIFF (Tag Image File Format) TIFF (Tag Image File Format)  Originally for scanners and frame grabbers  Used extensively on many platforms  Can be read/written by NIH Image  Supports lossless compression Reference: www.shortcourses.com/chapter07.htm

45. Common Image File Formats JPEG (“jay-peg” Joint Photographic Experts Group) JPEG (“jay-peg” Joint Photographic Experts Group)  Originally referred to a compression method but now refers to the associated file format with or without compression  Most common World Wide Web file format  Supports progressive display where an image is first displayed at low resolution and then at higher resolution.  Uses a lossy compression technique  Optimized for storing photographs and not as good for line art  Supports 24-bit color Reference: www.shortcourses.com/chapter07.htm

46. Common Image File Formats GIF (“jiff” Graphics Interchange Format) GIF (“jiff” Graphics Interchange Format)  Also widely used on the Web  Supports progressive display  Mostly used for line art as opposed to photographs  Only supports 8-bit color Reference: www.shortcourses.com/chapter07.htm