Digital image processing Chapter 7 Digital image processing Introduction to Remote Sensing Instructor: Dr. Cheng-Chien Liu Department of Earth Sciences National Cheng Kung University Last updated: 18 November 2004

Outline Introduction Image enhancement Image rectification and restoration Image classification Data merging Hyperspectral image analysis

Introduction Definition of digital image: f(x,y) Origin of DIP DN: Digital Number Processing Origin of DIP Applications of DIP Classified by EM spectrum g, X, UV, VNIR, IR, microwave, radiowave Our focus: VNIR Sound Ultrasound, … Components of DIP Contents of DIP

Enhancement Histogram Thresholding: Fig 7.11 Level slicing: Fig 7.12 Contrast stretching: Fig 7.13

Image rectification and restoration Rectification 糾正  distortion 畸變 Restoration 復原 degradation Source Digital image acquisition type Platform TFOV

Image rectification and restoration (cont.) Two-step procedure of geometric correction Systematic (predictable) e.g. eastward rotation of the earth  skew distortion Deskewing  offest each successive scan line slightly to the west  parallelogram image Random (unpredictable) e.g. random distortions and residual unknown systematic distortions Ground control points (GCPs) Highway intersections, distinct shoreline features,… Two coordinate transformation equations Distorted-image coordinate  Geometrically correct coordinate

Image rectification and restoration (cont.) Affine coordinate transform Six factors Transformation equation x = a0 + a1X + a2Y y = b0 + b1X + b2Y (x, y): image coordinate (X, Y): ground coordinate Six parameters  six conditions  3 GCPs If GCPs > 3  redundancy  LS solutions

Image rectification and restoration (cont.) Resampling Fig 7.1: Resampling process Transform coordinate Adjust DN value  perform after classification Methods Nearest neighbor Bilinear interpolation Bicubic convolution

Image rectification and restoration (cont.) Nearest neighbor Fig 7.1: a  a΄ (shaded pixel) Fig C.1: implement Rounding the computed coordinates to the nearest whole row and column number Advantage Computational simplicity Disadvantage Disjointed appearance: feature offset spatially up to ½ pixel (Fig 7.2b)

Image rectification and restoration (cont.) Bilinear interpolation Fig 7.1: a, b, b, b  a΄ (shaded pixel) Takes a distance-weighted average of the DNs of the four nearest pixels Fig C.2a: implement Eq. C.2 Eq. C.3 Advantage Smoother appearing (Fig 7.2c) Disadvantage Alter DN values Performed after image classification procedures

Image rectification and restoration (cont.) Bicubic (cubic) interpolation Fig 7.1: a, b, b, b, c, …  a΄ (shaded pixel) Takes a distance-weighted average of the DNs of the four nearest pixels Fig C.2b: implement Eq. C.5 Eq. C.6 Eq. C.7 Advantage (Fig 7.2d) Smoother appearing Provide a slightly sharper image than the bilinear interpolation image Disadvantage Alter DN values Performed after image classification procedures

Image rectification and restoration (cont.) Radiometric correction 輻射校正 Varies with sensors Mosaics of images taken at different times  require radiometric correction Influence factors Scene illumination Atmospheric correction Viewing geometry Instrument response characterstics

Image rectification and restoration (cont.) Sun elevation correction Fig 7.3: seasonal variation Normalize by calculating pixel brightness values assuming the sun was at the zenith on each date of sensing Multiply by cosq0 Earth-Sun distance correction Decrease as the square of the Earth-Sun distance Divided by d2 Combined influence

Image rectification and restoration (cont.) Atmospheric correction Atmospheric effects Attenuate (reduce) the illuminating energy Scatter and add path radiance Combination Haze compensation  minimize Lp Band of zero Lp (e.q.) NIR for clear water Path length compensation Off-nadir pixel values are normalized to their nadir equivalents

Image rectification and restoration (cont.) Conversion of DNs to radiance values Measure over time using different sensors Different range of reflectance e.g. land  water Fig 7.4: radiometric response function Linear Wavelength-dependent Characteristics are monitored using onboard calibration lamp DN = GL + B G: channel gain (slope) B: channel offset (intercept) Fig 7.5: inverse of radiometric response function Equation LMAX: saturated radiance LMAX - LMIN: dynamic range for the channel

Image rectification and restoration (cont.) Noise Definition Sources Periodic drift, malfunction of a detector, electronic interference, intermittent hiccups in the data transmission and recording sequence Influence Degrade or mask the information content

Image rectification and restoration (cont.) Systematic noise Striping or banding e.g. Landsat MSS six detectors drift Destriping (Fig 7.6) Compile a set of histograms Compare their mean and median values  identify the problematic detectors Gray-scale adjustment factors Line drop Line drop correction (Fig 7.7) Replace with values averaged from the above and below

Image rectification and restoration (cont.) Random noise Bit error  spikey  salt and pepper or snowy appearance Moving windows Fig 7.8: moving window Fig 7.9: an example of noise suppression algorithm Fig 7.10: application to a real imagey

Image classification Overall objective of classification Automatically categorize all pixels in an image into land cover classes or themes Three pattern recognitions Spectral pattern recognition  emphasize in this chapter Spatial pattern recognition Temporal pattern recognition Selection of classification No single “right” approach Depend on The nature of the data being analyzed The computational resources available The intended application of the classified data

Supervised classification Fig 7.37 A hypothetical example Five bands: B, G, R, NIR, TIR, Six land cover types: water, sand, forest, urban, corn, hay Three basic steps (Fig 7.38) Training stage Classification stage Output stage

Supervised classification (cont.) Classification stage Fig 7.39 Pixel observations from selected training sites plotted on scatter diagram Use two bands for demonstration, can be applied to any band number Clouds of points  multidimensional descriptions of the spectral response patterns of each category of cover type to be interpreted Minimum-Distance-to-Mean classifier Fig 7.40 Mean vector for each category Pt 1  Corn Pt 2  Sand ?!! Advantage: mathematically simple and computationally efficient Disadvantage: insensitive to different degrees of variance in the spectral response data Not widely used if the spectral classes are close to one another in the measurement space and have high variance

Supervised classification (cont.) Classification stage (cont.) Parallelepiped classifier Fig 7.41 Range for each category Pt 1  Hay ?!! Pt 2  Urban Advantage: mathematically simple and computationally efficient Disadvantage: confuse if correlation or high covariance are poorly described by the rectangular decision regions Positive covariance: Corn, Hay, Forest Negative covariance: Water Alleviate by use of stepped decision region boundaries (Fig 7.42)

Supervised classification (cont.) Classification stage (cont.) Gaussian maximum likelihood classifier Assumption: the distribution of the cloud of points is Gaussian distribution Probability density functions  mean vector and covariance matrix (Fig. 7.43) Fig 7.44: Ellipsoidal equiprobability contours Bayesian classifier A priori probability (anticipated likelihood of occurrence) Two weighting factors If suitable data exist for these factors, the Bayesian implementation of the classifier is preferable Disadvantage: computational efficiency Look-up table approach Reduce the dimensionality (principal or canonical components transform) Simplify classification computation by separate certain classes a prior Water is easier to separate by use of NIR/Red ratio

Supervised classification (cont.) Training stage Classification  automatic work Assembling the training data  manual work Both an art and a science Substantial reference data Thorough knowledge of the geographic area You are what you eat!  Results of classification are what you train! Training data Both representative and complete All spectral classes constituting each information class must be adequately represented in the training set statistics used to classify an image e.g. water (turbid or clear) e.g. crop (date, type, soil moisture, …) It is common to acquire data from 100+ training areas to represent the spectral variability

Supervised classification (cont.) Training stage (cont.) Training area Delineate boundaries (Fig 7.45) Carefully located boundaries  no edge pixels Seed pixel Choose seed pixel  statistically based criteria  contiguous pixels  cluster Training pixels Number At least n+1 pixels for n spectral bands In practice, 10n to 100n pixels is used Dispersion  representative  Training set refinement Make sure the sample size is sufficient Assess the overall quality Check if all data sets are normally distributed and spectrally pure Avoid redundancy Delete or merge

Supervised classification (cont.) Training stage (cont.) Training set refinement process Graphical representation of the spectral response patterns Fig 7.46: Histograms for data points included in the training areas of “hay” Visual check on the normality of the spectral response distribution Two subclasses: normal and bimodal Fig 7.47: Coincident spectral plot Corn/hay overlap for all bands Band 3 and 5 for hay/corn separation (use scatter plot) Fig 7.48: SPOT HRV multi-spectral images Fig 7.49 scatter plot of band 1 versus band 2 Fig 7.50 scatter plot of band 2 versus band 3  less correlated  adequate Quantitative expressions of category separation Transform divergence: a covariance-weighted distance between category means Table 7.1: Portion of a divergence matrix (<1500  spectrally similar classes)

Supervised classification (cont.) Training stage (cont.) Training set refinement process (cont.) Self-classification of training set data Error matrix  for training area not for the test area or the overall scene Tell us how well the classifier can classify the training areas and nothing more Overall accuracy is perform after the classification and output stage Interactive preliminary classification Plate 29: sample interactive preliminary classification procedure Representative subscene classification Complete the classification for the test area  verify and improve Summary Revise with merger, deletion and addition to form the final set of statistics used in classification Accept misclassification accuracy of a class that occurs rarely in the scene to preserve the accuracy over extensive areas Alternative methods for separating two spectrally similar classes  GIS data, visual interpretation, field check, multi-temporal or spatial pattern recognition procedures, …

Unsupervised classification Unsupervised  supervised Supervised  define useful information categories  examine their spectral separability Unsupervised  determine spectral classes  define their informational utility Illustration: Fig 7.51 Advantage: the spectral classes are found automatically (e.g. stressed class)

Unsupervised classification (cont.) Clustering algorithms K-means Locate centers of seed clusters  assign all pixels to the cluster with the closest mean vector  revise mean vectors for each clusters  reclassify the image  iterative until there is no significant change Iterative self-organizing data analysis (ISODATA) Permit the number of clusters to change from on iteration to the next by Merging: distance < some predefined minimum distance Splitting: standard deviation > some predefined maximum distance Deleting: pixel number in a cluster < some specified minimum number Texture/roughness Texture: the multidimensional variance observed in a moving window passed through the image Moving window  variance  threshold  smooth/rough

Unsupervised classification (cont.) Poor representation Roads and other linear features  not smooth Solution  hybrid classification Table 7.2 Outcome 1: ideal result Outcome 2: subclasses  classes Outcome 3: a more troublesome result The information categories is spectrally similar and cannot be differentiated in the given data set

Hybrid classification Unsupervised training areas Image sub-areas chosen intentionally to be quite different from supervised training areas Supervised  regions of homogeneous cover type Unsupervised  contain numerous cover types at various locations throughout the scene To identify the spectral classes Guided clustering Delineate training areas for class X Cluster all class X into spectral subclasses X1, X2, … Merge or delete class X signatures Repeat for all classes Examine all class signatures and merge/delete signatures Perform maximum likelihood classification Aggregate spectral subclasses

Classification of mixed pixels IFOV includes more than one type/feature Low resolution sensors  more serious Subpixel classification Spectral mixture analysis A deterministic method (not a statistical method) Pure reference spectral signatures Measured in the lab, in the field, or from the image itself Endmembers Basic assumption The spectral variation in an image is caused by mixtures of a limited number of surface materials Linear mixture  satisfy two basic conditions simultaneously The sum of the fractional proportions of all potential endmembers SFi = 1 The observed DNl for each pixel B band  B equations B+1 equations  solve B+1 endmember fractions Fig 7.52: example of a linear spectral mixture analysis Drawback: multiple scattering  nonlinear mixturemodel

Classification of mixed pixels (cont.) Subpixel classification (cont.) Fuzzy classification A given pixel may have partial membership in more than one category Fuzzy clustering Conceptually similar to the K-means unsupervised classification approach Hard boundaries  fuzzy regions Membership grade Fuzzy supervised classification A classified pixel is assigned a membership grade with respect to its membership in each information class

The output stage Image classification  output products  end users Graphic products Plate 30 Tabular data Digital information files

Postclassification smoothing Majority filter Fig 7.53 (a) original classification  salt-and-pepper appearance (b) 3 x 3 pixel-majority filter (c) 5 x 5 pixel-majority filter Spatial pattern recognition

Classification accuracy assessment Significance A classification is not complete until its accuracy is assessed Classification error matrix Error matrix (confusion matrix, contingency table) Table 7.3 Omission (exclusion) 漏授 Non-diagonal column elements (e.g. 16 sand pixels were omitted) Commission (inclusion) 誤授 Non-diagonal raw elements (e.g. 38 urban pixels + 79 hay pixels were included in corn) Overall accuracy Producer’s accuracy 生產者準確度 Indicate how well training set pixels of the given cover type are classified User’s accuracy 使用者準確度 Indicate the probability that a pixel classified into a given category actually represents that category on the ground Training area accuracies are sometimes used in the literature as an indication of overall accuracy. They should not be!

Classification accuracy assessment (cont.) Sampling considerations Test area Different and more extensive than training area Withhold some training areas for postclassification accuracy assessment Wall-to-wall comparison Expensive Defeat the whole purpose of remote sensing Random sampling Collect large sample of randomly distributed points  too expensive and difficult e.g. 3/4 of Taiwan area is covered by The Central mountain Only sample those pixels without influence of potential registration error Several pixels away from field boundaries Stratified random sampling Each land cover category  Stratum

Classification accuracy assessment (cont.) Sampling considerations (cont.) Sample unit Individual pixels, clusters of pixels or polygons Sample number General area: 50 samples per category Large area or more than 12 categories: 75 – 100 samples per category Depend on the variability of each category Wetland need more samples than open water

Data Merging and GIS Integration RS applications  data merging  unlimited variety of data Multi-resolution  data fusion Plate 1: GIS (soil erodibility + slope information) Trend Boundary between DIP and GIS  blurred Fully integrated spatial analysis systems  norm

Multi-temporal data merging Same area but different dates  composites  visual interpretation e.g. agricultural crop Plate 31(a): mapping invasive plant species NDVI from Landsat-7 ETM+ March 7  blue April 24  green October 15  red GIS-derived wetland boundary  eliminate the interpretation of false positive areas Plate 31(b): mapping of algae bloom Enhance the automated land cover classification Register all spectral bands from all dates into one master data set More data for classification Principal components analysis  reduce the dimensionality  manipulate, store, classify, … Multi-temporal profile Fig 7.54: greenness. (tp, s, Gm, G0)

Change detection procedures Types of interest Short term phenomena: e.g. snow cover, floodwater Long tern phenomena: e.g. urban fringe development, desertification Ideal conditions Same sensor, spatial resolution, viewing geometry, time of day An ideal orbit: ROCSAT-2 Anniversary dates Accurate spatial registration Environmental factors Lake level, tidal stage, wind, soil moisture condition, …

Change detection procedures (cont.) Approach Post-classification comparison Independent classification and registration Change with dates Classification of multi-temporal data sets A single classification is performed on a combined data sets Great dimensionality and complexity  redundancy Principal components analysis Two or more images  one multiband image Uncorrelated principal components  areas of change Difficult to interpret or identify the change Plate 32: (a) before (b) after (c) principal component image

Change detection procedures (cont.) Approach (cont.) Temporal image differencing One image is subtracted from the other Change-no change threshold Add a constant to each difference image for display purpose Temporal image ratioing One image is divided by the other No change area  ratio  1 Change vector analysis Fig 7.55 Change-versus-no-change binary mask Traditional classification of time 1 image Two-band (one from time 1 and the other from time 2)  algebraic operation  threshold  binary mask  apply to time 2 image

Change detection procedures (cont.) Approach (cont.) Delta transformation Fig 7.56 (a): no spectral change between two dates (b): natural variability in the landscape between dates (c): effect of uniform atmospheric haze differences between dates (d): effect of sensor drift between dates (e): brighter or darker pixels indicate land cover change (f): delta transformation Fig 7.57: application of delta transformation to Landsat TM images of forest

Multisensor image merging Plate 33: IHS multisensor image merger of SPOT HRV, landsat TM and digital orthophoto data Multi-spectral scanner + radar image data

Merging of image data with ancillary data Image + DEM  synthetic stereoscopic images Fig 7.58: synthetic stereopari generated from a single Landsat MSS image and a DEM Standard Landsat images  fixed, weak stereoscopic effect in the relatively small areas of overlap between orbit passes Produce perspective-view images Fig 7.59: perspective-view image of Mount Fuji

Incorporating GIS data in automated land cover classification Useful GIS data (ancillary data) Soil types, census statistics, ownership boundaries, zoning districts, … Geographic stratification Ancillary data  geographic stratification  classification Basis of stratification Single variable: upland  wetland, urban  rural Factors: landscape units or ecoregions that combine several interrelated variables (e.g. local climate, soil type, vegetation, landform)

Incorporating GIS data in automated land cover classification (cont.) Multi-source image classification decision rules (user-defined) Plate 34: a composite land cover classification A supervised classification of TM image in early May A supervised classification of TM image in late June A supervised classification of both dates combined using a PCA A wetlands GIS layer A road DLG (digital line graph) Table 7.5: basis for sample decision rules