1. Computational Biology, Part 23 Biological Imaging II Robert F. Murphy Copyright  1996, 1999, 2000-2006. All rights reserved.

2. Outline Image Display Image Display Image Processing Image Processing Image Analysis Image Analysis Image Interpretation Image Interpretation

3. From Images to Knowledge Image Processing Image Image Analysis Image Image Interpretation Image Numbers Knowledge

4. Image Display Operations that change display without changing image Operations that change display without changing image  LUT - grayscale or color  Contrast stretching Operations that change image Operations that change image  reversible  non-reversible (majority)

5. Image Display - LUT change

7. Image Display - Enhance contrast

8. Image Display After enhancement uses full range Original (before contrast enhancement)

9. Thresholding Thresholding refers to the division of the pixels of an image into two classes: those below a certain value (the threshold) and those at or above it. The two classes are often shown in white and black, respectively. Thresholding refers to the division of the pixels of an image into two classes: those below a certain value (the threshold) and those at or above it. The two classes are often shown in white and black, respectively. Thresholding serves as a means to consider only a subset of the pixels of an images. Thresholding serves as a means to consider only a subset of the pixels of an images.

10. Thresholding The choice of threshold must be made empirically by considering the nature of the sample, the type and number of objects expected in the image, and/or a histogram of pixel values The choice of threshold must be made empirically by considering the nature of the sample, the type and number of objects expected in the image, and/or a histogram of pixel values The threshold can be specified as a multiple of the background value (normally the most common value) for partial automation The threshold can be specified as a multiple of the background value (normally the most common value) for partial automation

11. Thresholding

12. Once a threshold has been applied, the resulting image may be Once a threshold has been applied, the resulting image may be  displayed in black and white  displayed with above threshold pixels at their original intensities and below threshold pixels in black

13. Thresholding Once a threshold has been applied, the resulting image may be Once a threshold has been applied, the resulting image may be  saved as a new image with only pixels above threshold being retained (others set to 0)  saved as or converted to a binary image (above threshold pixels set to 1, below threshold pixels set to 0)

14. Binary image operations Erosion Erosion  Remove pixels from edges of objects  Set “on” pixel to “off” if four or more of its eight neighbors are white Dilation Dilation  Add pixels to edges of objects  Set “off” pixel to “on” if four or more of its neighbors are black

15. Binary image operations Process/Binary/Threshold does auto threshold and applies it to make binary image

16. Binary image operations

17. This image shows just the pixels that were turned off by the erode operation

18. Binary image operations Open Open  Smooth objects and fill in small holes  Erosion followed by dilation Close Close  Smooth objects and fill in small holes  Dilation followed by erosion

19. Binary image operations Outline Outline  Find “on” pixel, trace around outside until return to first “on” pixel Skeletonize Skeletonize  Remove pixels from the edges of objects until the objects are one pixel wide

20. Binary image operations

21. Binary image operations - outline

22. Basic Image Processing Operations Image Math Image Math Kernel/Filter Operations Kernel/Filter Operations Image Calculator Image Calculator

23. Arithmetic Operations Two cases: Two cases:  Perform a single operand operation (e.g., logarithm, square root) on each pixel of an image  Perform a dual operand operation (e.g., add, multiply) on each pixel of an image using a constant as the second operand In both cases, the result is usually stored in the same pixel location (“storing in place”) In both cases, the result is usually stored in the same pixel location (“storing in place”)

24. Arithmetic Operations

25. Kernel/Filter Operations Basic idea: Use a matrix (usually square and of odd dimension, e.g., 3x3) in combination with an image to generate a new image Basic idea: Use a matrix (usually square and of odd dimension, e.g., 3x3) in combination with an image to generate a new image Algorithm: Algorithm:  For each pixel in the image (the current pixel)  Align the matrix to center it on that pixel  For each position in the matrix, multiply the corresponding pixel value in the image by the value in the matrix and sum the results  Store the result in the current pixel

26. Kernel/Filter Operations A matrix used in this fashion is called a kernel or filter A matrix used in this fashion is called a kernel or filter Note that the operation is different from matrix multiplication of the kernel by the image because Note that the operation is different from matrix multiplication of the kernel by the image because  the dimensions don’t match, and  all elements of the matrix are combined to give one result

27. Common Kernel Operations used in Image Processing Smoothing Smoothing Sharpening Sharpening Edge Finding Edge Finding

28. Original image Original image

29. Examples of Kernel Operations using NIH Image Smooth Smooth

30. Results of one Smooth Results of one Smooth

31. Results of a second Smooth Results of a second Smooth

32. Examples of Kernel Operations using NIH Image Close smoothed image, reopen original image, then Sharpen Close smoothed image, reopen original image, then Sharpen

33. Original image Original image

34. Image after one Sharpen Image after one Sharpen

35. Image after a second Sharpen Image after a second Sharpen

36. Examples of Kernel Operations using NIH Image Close sharpened image, reopen original image, then Find Edges Close sharpened image, reopen original image, then Find Edges

37. Image after Find Edges Image after Find Edges

38. Example kernels Smoothing Smoothing

39. Example kernels Sharpen Sharpen

40. Example kernels Edge detection (Sobel) Edge detection (Sobel)

41. Image Math Basic idea: Combine two images using an dual operand operator to generate a new image Basic idea: Combine two images using an dual operand operator to generate a new image Algorithm: Algorithm:  For each pixel in the first image, operate on it using the corresponding pixel in the second image and store the result in the corresponding pixel in a new (output) image

42. Image Math Any operator can be used Any operator can be used Most common operators: Most common operators:  division: generate ratio image  logical AND: mask one image with another (usually binary) image

43. Examples of Image Math using NIH Image Open original image and sharpen once (save as Abdomen.sharpen1), reopen original image Open original image and sharpen once (save as Abdomen.sharpen1), reopen original image

44. Ratio of sharp to original image (shows regions affected by sharpen) Ratio of sharp to original image (shows regions affected by sharpen)

45. Image Math vs. Arithmetic Operations Note difference between Image Math which does an operation on two images and Arithmetic which does an operation on a single image and a constant Note difference between Image Math which does an operation on two images and Arithmetic which does an operation on a single image and a constant

46. Summary: Basic Image Processing Operations Arithmetic Operations Arithmetic Operations  Inputs: Image, Constant (optional)  Common use: Subtract background Kernel Operations Kernel Operations  Inputs: Image, Kernel  Common use: Smoothing Image Math Image Math  Inputs: Two images  Common use: Generate ratio image

47. Image Processing Basic example of image processing: find objects in an image and describe them numerically Basic example of image processing: find objects in an image and describe them numerically

48. Object finding (Particle analysis) Principle: Identify a contiguous set of pixels that are all above some threshold Principle: Identify a contiguous set of pixels that are all above some threshold Implementation: Implementation:  Start with a binary (thresholded) image  Find a pixel that is “on” and start a list or map  Recursively search all nearest neighbors for additional pixels that are on and add them to the list or map

49. Object finding (Particle analysis) Start with a thresholded image (Image/Adjust/Threshold)

50. Object finding (Particle analysis)

51. Save as Excel file using Save As...

52. Object finding (Particle analysis) Uses: Uses:  Counting objects  Obtaining area measurements for objects  Obtaining integrated intensity  Isolating objects for other processing