1. Digital image processing
Chapter 7
Digital image processing
Introduction to Remote Sensing
Instructor: Dr. Cheng-Chien Liu
Department of Earth Sciences
National Cheng Kung University
Last updated: 18 November 2004

2. Outline Introduction Image enhancement
Image rectification and restoration
Image classification
Data merging
Hyperspectral image analysis

3. Introduction Definition of digital image: f(x,y) Origin of DIP
DN: Digital Number
Processing
Origin of DIP
Applications of DIP
Classified by EM spectrum
g, X, UV, VNIR, IR, microwave, radiowave
Our focus: VNIR
Sound
Ultrasound, …
Components of DIP
Contents of DIP

4. Enhancement Histogram Thresholding: Fig 7.11 Level slicing: Fig 7.12
Contrast stretching: Fig 7.13

5. Image rectification and restoration
Rectification 糾正  distortion 畸變
Restoration 復原 degradation
Source
Digital image acquisition type
Platform
TFOV

6. Image rectification and restoration (cont.)
Two-step procedure of geometric correction
Systematic (predictable)
e.g. eastward rotation of the earth  skew distortion
Deskewing  offest each successive scan line slightly to the west  parallelogram image
Random (unpredictable)
e.g. random distortions and residual unknown systematic distortions
Ground control points (GCPs)
Highway intersections, distinct shoreline features,…
Two coordinate transformation equations
Distorted-image coordinate  Geometrically correct coordinate

7. Image rectification and restoration (cont.)
Affine coordinate transform
Six factors
Transformation equation
x = a0 + a1X + a2Y
y = b0 + b1X + b2Y
(x, y): image coordinate
(X, Y): ground coordinate
Six parameters  six conditions  3 GCPs
If GCPs > 3  redundancy  LS solutions

8. Image rectification and restoration (cont.)
Resampling
Fig 7.1: Resampling process
Transform coordinate
Adjust DN value  perform after classification
Methods
Nearest neighbor
Bilinear interpolation
Bicubic convolution

9. Image rectification and restoration (cont.)
Nearest neighbor
Fig 7.1: a  a΄ (shaded pixel)
Fig C.1: implement
Rounding the computed coordinates to the nearest whole row and column number
Advantage
Computational simplicity
Disadvantage
Disjointed appearance: feature offset spatially up to ½ pixel (Fig 7.2b)

10. Image rectification and restoration (cont.)
Bilinear interpolation
Fig 7.1: a, b, b, b  a΄ (shaded pixel)
Takes a distance-weighted average of the DNs of the four nearest pixels
Fig C.2a: implement
Eq. C.2
Eq. C.3
Advantage
Smoother appearing (Fig 7.2c)
Disadvantage
Alter DN values
Performed after image classification procedures

11. Image rectification and restoration (cont.)
Bicubic (cubic) interpolation
Fig 7.1: a, b, b, b, c, …  a΄ (shaded pixel)
Takes a distance-weighted average of the DNs of the four nearest pixels
Fig C.2b: implement
Eq. C.5
Eq. C.6
Eq. C.7
Advantage (Fig 7.2d)
Smoother appearing
Provide a slightly sharper image than the bilinear interpolation image
Disadvantage
Alter DN values
Performed after image classification procedures

12. Image rectification and restoration (cont.)
Radiometric correction 輻射校正
Varies with sensors
Mosaics of images taken at different times  require radiometric correction
Influence factors
Scene illumination
Atmospheric correction
Viewing geometry
Instrument response characterstics

13. Image rectification and restoration (cont.)
Sun elevation correction
Fig 7.3: seasonal variation
Normalize by calculating pixel brightness values assuming the sun was at the zenith on each date of sensing
Multiply by cosq0
Earth-Sun distance correction
Decrease as the square of the Earth-Sun distance
Divided by d2
Combined influence

14. Image rectification and restoration (cont.)
Atmospheric correction
Atmospheric effects
Attenuate (reduce) the illuminating energy
Scatter and add path radiance
Combination
Haze compensation  minimize Lp
Band of zero Lp (e.q.) NIR for clear water
Path length compensation
Off-nadir pixel values are normalized to their nadir equivalents

15. Image rectification and restoration (cont.)
Conversion of DNs to radiance values
Measure over time using different sensors
Different range of reflectance
e.g. land  water
Fig 7.4: radiometric response function
Linear
Wavelength-dependent
Characteristics are monitored using onboard calibration lamp
DN = GL + B
G: channel gain (slope)
B: channel offset (intercept)
Fig 7.5: inverse of radiometric response function
Equation
LMAX: saturated radiance
LMAX - LMIN: dynamic range for the channel

16. Image rectification and restoration (cont.)
Noise
Definition
Sources
Periodic drift, malfunction of a detector, electronic interference, intermittent hiccups in the data transmission and recording sequence
Influence
Degrade or mask the information content

17. Image rectification and restoration (cont.)
Systematic noise
Striping or banding
e.g. Landsat MSS six detectors drift
Destriping (Fig 7.6)
Compile a set of histograms
Compare their mean and median values  identify the problematic detectors
Gray-scale adjustment factors
Line drop
Line drop correction (Fig 7.7)
Replace with values averaged from the above and below

18. Image rectification and restoration (cont.)
Random noise
Bit error  spikey  salt and pepper or snowy appearance
Moving windows
Fig 7.8: moving window
Fig 7.9: an example of noise suppression algorithm
Fig 7.10: application to a real imagey

19. Image classification Overall objective of classification
Automatically categorize all pixels in an image into land cover classes or themes
Three pattern recognitions
Spectral pattern recognition  emphasize in this chapter
Spatial pattern recognition
Temporal pattern recognition
Selection of classification
No single “right” approach
Depend on
The nature of the data being analyzed
The computational resources available
The intended application of the classified data

20. Supervised classification
Fig 7.37
A hypothetical example
Five bands: B, G, R, NIR, TIR,
Six land cover types: water, sand, forest, urban, corn, hay
Three basic steps (Fig 7.38)
Training stage
Classification stage
Output stage

21. Supervised classification (cont.)
Classification stage
Fig 7.39
Pixel observations from selected training sites plotted on scatter diagram
Use two bands for demonstration, can be applied to any band number
Clouds of points  multidimensional descriptions of the spectral response patterns of each category of cover type to be interpreted
Minimum-Distance-to-Mean classifier
Fig 7.40
Mean vector for each category
Pt 1  Corn
Pt 2  Sand ?!!
Advantage: mathematically simple and computationally efficient
Disadvantage: insensitive to different degrees of variance in the spectral response data
Not widely used if the spectral classes are close to one another in the measurement space and have high variance

22. Supervised classification (cont.)
Classification stage (cont.)
Parallelepiped classifier
Fig 7.41
Range for each category
Pt 1  Hay ?!!
Pt 2  Urban
Advantage: mathematically simple and computationally efficient
Disadvantage: confuse if correlation or high covariance are poorly described by the rectangular decision regions
Positive covariance: Corn, Hay, Forest
Negative covariance: Water
Alleviate by use of stepped decision region boundaries (Fig 7.42)

23. Supervised classification (cont.)
Classification stage (cont.)
Gaussian maximum likelihood classifier
Assumption: the distribution of the cloud of points is Gaussian distribution
Probability density functions  mean vector and covariance matrix (Fig. 7.43)
Fig 7.44: Ellipsoidal equiprobability contours
Bayesian classifier
A priori probability (anticipated likelihood of occurrence)
Two weighting factors
If suitable data exist for these factors, the Bayesian implementation of the classifier is preferable
Disadvantage: computational efficiency
Look-up table approach
Reduce the dimensionality (principal or canonical components transform)
Simplify classification computation by separate certain classes a prior
Water is easier to separate by use of NIR/Red ratio

24. Supervised classification (cont.)
Training stage
Classification  automatic work
Assembling the training data  manual work
Both an art and a science
Substantial reference data
Thorough knowledge of the geographic area
You are what you eat!  Results of classification are what you train!
Training data
Both representative and complete
All spectral classes constituting each information class must be adequately represented in the training set statistics used to classify an image
e.g. water (turbid or clear)
e.g. crop (date, type, soil moisture, …)
It is common to acquire data from 100+ training areas to represent the spectral variability

25. Supervised classification (cont.)
Training stage (cont.)
Training area
Delineate boundaries (Fig 7.45)
Carefully located boundaries  no edge pixels
Seed pixel
Choose seed pixel  statistically based criteria  contiguous pixels  cluster
Training pixels
Number
At least n+1 pixels for n spectral bands
In practice, 10n to 100n pixels is used
Dispersion  representative 
Training set refinement
Make sure the sample size is sufficient
Assess the overall quality
Check if all data sets are normally distributed and spectrally pure
Avoid redundancy
Delete or merge

26. Supervised classification (cont.)
Training stage (cont.)
Training set refinement process
Graphical representation of the spectral response patterns
Fig 7.46: Histograms for data points included in the training areas of “hay”
Visual check on the normality of the spectral response distribution
Two subclasses: normal and bimodal
Fig 7.47: Coincident spectral plot
Corn/hay overlap for all bands
Band 3 and 5 for hay/corn separation (use scatter plot)
Fig 7.48: SPOT HRV multi-spectral images
Fig 7.49 scatter plot of band 1 versus band 2
Fig 7.50 scatter plot of band 2 versus band 3  less correlated  adequate
Quantitative expressions of category separation
Transform divergence: a covariance-weighted distance between category means
Table 7.1: Portion of a divergence matrix (<1500  spectrally similar classes)

27. Supervised classification (cont.)
Training stage (cont.)
Training set refinement process (cont.)
Self-classification of training set data
Error matrix  for training area not for the test area or the overall scene
Tell us how well the classifier can classify the training areas and nothing more
Overall accuracy is perform after the classification and output stage
Interactive preliminary classification
Plate 29: sample interactive preliminary classification procedure
Representative subscene classification
Complete the classification for the test area  verify and improve
Summary
Revise with merger, deletion and addition to form the final set of statistics used in classification
Accept misclassification accuracy of a class that occurs rarely in the scene to preserve the accuracy over extensive areas
Alternative methods for separating two spectrally similar classes  GIS data, visual interpretation, field check, multi-temporal or spatial pattern recognition procedures, …

28. Unsupervised classification
Unsupervised  supervised
Supervised  define useful information categories  examine their spectral separability
Unsupervised  determine spectral classes  define their informational utility
Illustration: Fig 7.51
Advantage: the spectral classes are found automatically (e.g. stressed class)

29. Unsupervised classification (cont.)
Clustering algorithms
K-means
Locate centers of seed clusters  assign all pixels to the cluster with the closest mean vector  revise mean vectors for each clusters  reclassify the image  iterative until there is no significant change
Iterative self-organizing data analysis (ISODATA)
Permit the number of clusters to change from on iteration to the next by
Merging: distance < some predefined minimum distance
Splitting: standard deviation > some predefined maximum distance
Deleting: pixel number in a cluster < some specified minimum number
Texture/roughness
Texture: the multidimensional variance observed in a moving window passed through the image
Moving window  variance  threshold  smooth/rough

30. Unsupervised classification (cont.)
Poor representation
Roads and other linear features  not smooth
Solution  hybrid classification
Table 7.2
Outcome 1: ideal result
Outcome 2: subclasses  classes
Outcome 3: a more troublesome result
The information categories is spectrally similar and cannot be differentiated in the given data set

31. Hybrid classification
Unsupervised training areas
Image sub-areas chosen intentionally to be quite different from supervised training areas
Supervised  regions of homogeneous cover type
Unsupervised  contain numerous cover types at various locations throughout the scene
To identify the spectral classes
Guided clustering
Delineate training areas for class X
Cluster all class X into spectral subclasses X1, X2, …
Merge or delete class X signatures
Repeat for all classes
Examine all class signatures and merge/delete signatures
Perform maximum likelihood classification
Aggregate spectral subclasses

32. Classification of mixed pixels
IFOV includes more than one type/feature
Low resolution sensors  more serious
Subpixel classification
Spectral mixture analysis
A deterministic method (not a statistical method)
Pure reference spectral signatures
Measured in the lab, in the field, or from the image itself
Endmembers
Basic assumption
The spectral variation in an image is caused by mixtures of a limited number of surface materials
Linear mixture  satisfy two basic conditions simultaneously
The sum of the fractional proportions of all potential endmembers SFi = 1
The observed DNl for each pixel
B band  B equations
B+1 equations  solve B+1 endmember fractions
Fig 7.52: example of a linear spectral mixture analysis
Drawback: multiple scattering  nonlinear mixturemodel

33. Classification of mixed pixels (cont.)
Subpixel classification (cont.)
Fuzzy classification
A given pixel may have partial membership in more than one category
Fuzzy clustering
Conceptually similar to the K-means unsupervised classification approach
Hard boundaries  fuzzy regions
Membership grade
Fuzzy supervised classification
A classified pixel is assigned a membership grade with respect to its membership in each information class

34. The output stage Image classification  output products  end users
Graphic products
Plate 30
Tabular data
Digital information files

35. Postclassification smoothing
Majority filter
Fig 7.53
(a) original classification  salt-and-pepper appearance
(b) 3 x 3 pixel-majority filter
(c) 5 x 5 pixel-majority filter
Spatial pattern recognition

36. Classification accuracy assessment
Significance
A classification is not complete until its accuracy is assessed
Classification error matrix
Error matrix (confusion matrix, contingency table)
Table 7.3
Omission (exclusion) 漏授
Non-diagonal column elements (e.g. 16 sand pixels were omitted)
Commission (inclusion) 誤授
Non-diagonal raw elements (e.g. 38 urban pixels + 79 hay pixels were included in corn)
Overall accuracy
Producer’s accuracy 生產者準確度
Indicate how well training set pixels of the given cover type are classified
User’s accuracy 使用者準確度
Indicate the probability that a pixel classified into a given category actually represents that category on the ground
Training area accuracies are sometimes used in the literature as an indication of overall accuracy. They should not be!

37. Classification accuracy assessment (cont.)
Sampling considerations
Test area
Different and more extensive than training area
Withhold some training areas for postclassification accuracy assessment
Wall-to-wall comparison
Expensive
Defeat the whole purpose of remote sensing
Random sampling
Collect large sample of randomly distributed points  too expensive and difficult
e.g. 3/4 of Taiwan area is covered by The Central mountain
Only sample those pixels without influence of potential registration error
Several pixels away from field boundaries
Stratified random sampling
Each land cover category  Stratum

38. Classification accuracy assessment (cont.)
Sampling considerations (cont.)
Sample unit
Individual pixels, clusters of pixels or polygons
Sample number
General area: 50 samples per category
Large area or more than 12 categories: 75 – 100 samples per category
Depend on the variability of each category
Wetland need more samples than open water

39. Data Merging and GIS Integration
RS applications  data merging  unlimited variety of data
Multi-resolution  data fusion
Plate 1: GIS (soil erodibility + slope information)
Trend
Boundary between DIP and GIS  blurred
Fully integrated spatial analysis systems  norm

40. Multi-temporal data merging
Same area but different dates  composites  visual interpretation
e.g. agricultural crop
Plate 31(a): mapping invasive plant species
NDVI from Landsat-7 ETM+
March 7  blue
April 24  green
October 15  red
GIS-derived wetland boundary  eliminate the interpretation of false positive areas
Plate 31(b): mapping of algae bloom
Enhance the automated land cover classification
Register all spectral bands from all dates into one master data set
More data for classification
Principal components analysis  reduce the dimensionality  manipulate, store, classify, …
Multi-temporal profile
Fig 7.54: greenness. (tp, s, Gm, G0)

41. Change detection procedures
Types of interest
Short term phenomena: e.g. snow cover, floodwater
Long tern phenomena: e.g. urban fringe development, desertification
Ideal conditions
Same sensor, spatial resolution, viewing geometry, time of day
An ideal orbit: ROCSAT-2
Anniversary dates
Accurate spatial registration
Environmental factors
Lake level, tidal stage, wind, soil moisture condition, …

42. Change detection procedures (cont.)
Approach
Post-classification comparison
Independent classification and registration
Change with dates
Classification of multi-temporal data sets
A single classification is performed on a combined data sets
Great dimensionality and complexity  redundancy
Principal components analysis
Two or more images  one multiband image
Uncorrelated principal components  areas of change
Difficult to interpret or identify the change
Plate 32: (a) before (b) after (c) principal component image

43. Change detection procedures (cont.)
Approach (cont.)
Temporal image differencing
One image is subtracted from the other
Change-no change threshold
Add a constant to each difference image for display purpose
Temporal image ratioing
One image is divided by the other
No change area  ratio  1
Change vector analysis
Fig 7.55
Change-versus-no-change binary mask
Traditional classification of time 1 image
Two-band (one from time 1 and the other from time 2)  algebraic operation  threshold  binary mask  apply to time 2 image

44. Change detection procedures (cont.)
Approach (cont.)
Delta transformation
Fig 7.56
(a): no spectral change between two dates
(b): natural variability in the landscape between dates
(c): effect of uniform atmospheric haze differences between dates
(d): effect of sensor drift between dates
(e): brighter or darker pixels indicate land cover change
(f): delta transformation
Fig 7.57: application of delta transformation to Landsat TM images of forest

45. Multisensor image merging
Plate 33: IHS multisensor image merger of SPOT HRV, landsat TM and digital orthophoto data
Multi-spectral scanner + radar image data

46. Merging of image data with ancillary data
Image + DEM
 synthetic stereoscopic images
Fig 7.58: synthetic stereopari generated from a single Landsat MSS image and a DEM
Standard Landsat images  fixed, weak stereoscopic effect in the relatively small areas of overlap between orbit passes
Produce perspective-view images
Fig 7.59: perspective-view image of Mount Fuji

47. Incorporating GIS data in automated land cover classification
Useful GIS data (ancillary data)
Soil types, census statistics, ownership boundaries, zoning districts, …
Geographic stratification
Ancillary data  geographic stratification  classification
Basis of stratification
Single variable: upland  wetland, urban  rural
Factors: landscape units or ecoregions that combine several interrelated variables (e.g. local climate, soil type, vegetation, landform)

48. Incorporating GIS data in automated land cover classification (cont.)
Multi-source image classification decision rules (user-defined)
Plate 34: a composite land cover classification
A supervised classification of TM image in early May
A supervised classification of TM image in late June
A supervised classification of both dates combined using a PCA
A wetlands GIS layer
A road DLG (digital line graph)
Table 7.5: basis for sample decision rules