1. EEE 498/591- Real-Time DSP1 What is image processing? x(t 1,t 2 ) : ANALOG SIGNAL x : real value (t 1,t 2 ) : pair of real continuous space (time) variables x(n 1,n 2 ) : DISCRETE SIGNAL (DIGITAL) x : discrete (quantized) real or integer value (n 1,n 2 ) : pair of integer indices CAMERA Analog Image x(t1,t2)x(t1,t2) DIGITIZER STORAGEPROCESS Display Perform analysis Reconstruct x(t 1,t 2 ) x(n1,n2)x(n1,n2) Sampling + Quantization Sensor with RGB color filters

2. EEE 498/591- Real-Time DSP2 Examples Sampled Black & White Photograph: x(n 1,n 2 ) x (n 1,n 2 ) scalar indicating piel intensity at location (n 1,n 2 ) For example: x = 0 Black x = 1 White 0 < x < 1 In-between Sampled color video/TV signal x R (n 1, n 2, n 3 ) x G (n 1, n 2, n 3 ) x B (n 1, n 2, n 3 )

3. EEE 498/591- Real-Time DSP3 Examples

4. EEE 498/591- Real-Time DSP4 How do we process images? Use DSP concepts as tools Exploit visual perception properties

5. EEE 498/591- Real-Time DSP5 Visual Perception We represent pixels as amplitude values (gray scale). 256 levels 1 0 128 levels 1 0 64 levels 1 0 32 levels 1 0 How much to sample (quantize) the gray scale? Humans can distinguish in the order of 100 levels of gray (about 40 to 100).

6. EEE 498/591- Real-Time DSP6 Visual Perception: Examples Original image, 8 bits per pixelProcessed image, 0.35 bits per pixel

7. EEE 498/591- Real-Time DSP7 Visual Perception RMSE = 8.5RMSE = 9.0

8. EEE 498/591- Real-Time DSP8 Image Enhancement  Objective: accentuate or improve appearance of features, for subsequent analysis or display (possibly, but not necessarily degraded by some phenomenon).  Examples of features: edges, boundaries, dynamic range and contrast.  Examples of applications: TV: enhance image for viewer (image quality, intelligibility, visual appearance). Preprocessing for machine identification.  Enhancement is not necessarily needed because of degradation but can be used possibly to remove degradation.

9. EEE 498/591- Real-Time DSP9 Image Enhancement Low contrast or dynamic range Blurred or faint edges Noise Modify low dynamic range    Remove noise Sharpen faint or blurred edges

10. EEE 498/591- Real-Time DSP10 Image Enhancement Types of distortions to correct:  Blur (Blurring due to camera motion, defocusing, …)  Noise (assumed to be additive often for simplicity, although not necessarily the case).  Contrast  Blocking artifacts in block-based transform coders.  Examples: Space photography Underwater photography Film grain noise

11. EEE 498/591- Real-Time DSP11 p(x)p(x) x a b Number of occurrences of pixel with a particular intensity x Contrast stretching  Degradation is commonly due to poor lighting.  Image probability distribution function (pdf) has narrow peak  poor contrast.  Image intensities are clustered in a small region  available dynamic range is not very well utilized. Contrast and Dynamic Range Modification

12. EEE 498/591- Real-Time DSP12 Contrast and Dynamic Range Modification  Possible solution: increase overall dynamic range  resulting image would appear to have a grater contrast  expand the amplitudes from a to b to cover available intensity range. p(x new ) x new

13. EEE 498/591- Real-Time DSP13 Contrast and Dynamic Range Modification  Idea: Gray scale or intensity level of an input image x(n 1,n 2 ) is modified according to a specific transformation (function) f(·). Note: f(·) is usually constrained to be a monotonically non-decreasing function of x  ensures that a pixel with higher intensity than another will not became a pixel with a lower intensity in output image x new. Typical stretching operator: abx x new a’ b’ L slope  slope  slope 

14. EEE 498/591- Real-Time DSP14 Contrast and Dynamic Range Modification Specific desired transformation depends on the application Example: compensation of display non-linearity  most suitable transformation depends on display non-linearity. In most applications, a good or suitable transformation can be identified by computing and analyzing the histogram of the input image to be enhanced. –The histogram is a scaled version of the image pdf. –The histogram gives pdf when scaled by the total number of pixels in the image.

15. EEE 498/591- Real-Time DSP15 Histogram Modification and Equalization Definition: The histogram of an image h(x) represents the number of pixels that have a specific intensity x  number of pixels as a function of intensity x. h(x) = scaled version of pdf p(x) Normalization ensures that normalized

16. EEE 498/591- Real-Time DSP16 Histogram Modification and Equalization Remarks:  Histogram modification methods popular because computing and modifying histogram of an image requires little computations.  Experienced person can easily determine needed transformation by analyzing histogram characteristics. But if too many images  automatic method is desired.  For typical natural images, the desired histogram has a maximum around the middle of the dynamic range and decreases slowly as the intensity increases or decreases.  Problem: Determine f(·) such that h output (x new ) = h d (x new ) x hi(x)hi(x) x new h d (x new ) L min L max x new = f(x) Desired histogram of output image

17. EEE 498/591- Real-Time DSP17 Histogram Modification and Equalization Histogram equalization: special case of histogram modification where h d (x) = constant  For a 256  256 image, with 256 intensity levels:  How can we do assignment? x new hd(x)hd(x) L min L max const Redistribute pixels by assigning pixels uniformly to the given levels. pixels assigned to each level

18. EEE 498/591- Real-Time DSP18 Histogram Modification and Equalization Method 1: Collect and redistribute pixels Example: 128 pixels at 8 levels 0L10L1 1L21L2 2L32L3 3L43L4 4L54L5 5L65L6 6L76L7 7L87L8 1 7 21 35 21 7 1 x h(x)h(x) 16 Collect and redistribute pixels. Get bundles of 16 regardless of their position in image. 0L10L1 1L21L2 2L32L3 3L43L4 4L54L5 5L65L6 6L76L7 7L87L8 x ho(x)ho(x) 16

19. EEE 498/591- Real-Time DSP19 Histogram Modification and Equalization  How to group them? One approach is to split bins  can choose pixels within a bin randomly  poor results that tend to be noisy. Compute average values of neighboring pixels and match to closest average  better, but still noisy. A better approach is to use a cumulative method.

20. EEE 498/591- Real-Time DSP20 Histogram Modification and Equalization Cumulative method for histogram equalization  Histogram equalization  desired histogram is constant at all levels.  Problem: Find transformation x o =f(x i ) such that that h output (x o ) = const If normalized histogram   uniform distribution x o = f(x i ) xixi hi(xi )hi(xi ) x min x max xoxo ho(xo )ho(xo ) 0L -1 const

21. EEE 498/591- Real-Time DSP21 Histogram Modification and Equalization  Solution: Compute input pdf Choose Why? if  y uniformly distributed between (0,1)  histogram uniformly distributed  need also to scale y because y  (0,1) instead of (0,L -1 ) or (L min,L max ) = normalized histogram = cumulative probability distribution of x i + scaling needed

22. EEE 498/591- Real-Time DSP22 Histogram Modification and Equalization Note: if  y uniformly distributed between (0,1) Proof:

23. EEE 498/591- Real-Time DSP23 Histogram Modification and Equalization Since x i is a discrete variable, integral is replaced by summation:  y min not necessarily 0 since Scaling can be done as follows only approximately uniformly distributed (because of discretization)

24. EEE 498/591- Real-Time DSP24 Histogram Modification and Equalization If input and output range from 0 to L – procedure can be described as follows: Procedure: x k = k; k=0,…, L = input amplitude levels y k ; k=0,…, L = output amplitude levels 1.Compute the histogram of the image to be improved. 2.Normalize histogram; Normalize amplitudes so that the sum of all values is equal to one and you have a pdf, p i (·). 3.Compute 4.Move bins in x k to locations in y k. 5.Scale y k to desired amplitude range (linear mapping). where- rounding operation

25. EEE 498/591- Real-Time DSP25 Histogram Modification and Equalization Example: 0.05 0.10 0.15 0.20 0.25 pi(xk)pi(xk) x k ; 0≤x k ≤7 0.14 0.25 0.2 0.15 0.1 0.07 0.05 0.04 1 2 3 4 5 670 L = 7 Find y k such that x k maps into y k :

26. EEE 498/591- Real-Time DSP26 Histogram Modification and Equalization y k ; 0≤y k ≤7 0.05 0.10 0.15 0.20 0.25 po(yk)po(yk) 0.14 0.25 0.2 0.16 1 3 5 670 0.25

27. EEE 498/591- Real-Time DSP27 Histogram Modification and Equalization Original Image Equalized Image

28. EEE 498/591- Real-Time DSP28 Thresholding Thresholding can be used to extract objects from image (segmentation) Histogram statistics can be used to define single or multiple thresholds to classify an image pixel-by-pixel. A simple approach:  Bimodal histogram  set the threshold to gray value corresponding to the deepest point in the histogram value.  Multimodal histogram  set thresholds to correspond to points in “valleys” of histogram.  Classify every pixel f(x,y) by comparing its gray level to the selected threshold.

29. EEE 498/591- Real-Time DSP29 Pixel-based direct classification methods Original image Image histogram Thresholded, T=12 Thresholded, T=166 Thresholded, T=225

30. EEE 498/591- Real-Time DSP30 Thresholding