Presentation is loading. Please wait.

Presentation is loading. Please wait.

Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035.

Similar presentations


Presentation on theme: "Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035."— Presentation transcript:

1 page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) /07/2002

2 page 2 of 50 Outline Introduction of Satellite Remote Sensing Remote Sensing Image Classification Methodology Data and Experiments Discussion and Conclusion

3 page 3 of 50 Earth Surface and Land Cover Map How to achieve an accurate land cover map in natural resources research?

4 page 4 of 50 Remote Sensing Remote Sensing is defined as: The science of acquiring, processing and interpreting images that record the interaction between electromagnetic energy and matter.

5 page 5 of 50 Different Type of Land Use and Land cover and Spectral Characteristics Blue Band Red BandNear-infrared Band Green Band

6 page 6 of 50 From Digital Image to Land-use Map Visualization Classification Multi-spectral Digital Image Pseudo Color Image Land-use and Land-cover Map

7 page 7 of 50 Classification System In this study, we used the USGS classification with 10 land cover categories: 1. Turf/Grass 2. Barren land 3. Conifer forest 4. Deciduous forest 5. Mixed forest 6. Brush land 7. Urban land 8. Water 9. Non-forest wetland 10. Forest wetland

8 page 8 of 50 Remote Sensing Image Classification Statistical Methods – Supervised – Unsupervised Artificial Neural Network Approaches Rule-based...

9 page 9 of 50 Supervised Remote Sensing Image Classification Vegetation Water Soil Band 1 (0 ~ 255) Band 2 (0 ~ 255)

10 page 10 of 50 Classification Process Mapping Relationship Methods: Statistical classifier ANN-based classifier Observation space Solution space Landsat TM Band1 Band2 Band3 Band4 Band5 Band6 Band7 0~255 0~255 0~255 0~255 0~255 0~255 0~255 Category 1 Category.. Category … Category N Water wetland Forest Agri. Urban Residential Category: Forest (Pattern)

11 page 11 of 50 Statistical Methods - Need Gaussian (Normal) distribution on the input data which is required by Bayesian classifier. - Restrictions about the format of input data. Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Water Soil

12 page 12 of 50 Statistical Methods: Deciduous forest Conifer forest Forest Wetland Mixed forest Brush Land Non-forest Wetland How to find a boundary for the following patterns?

13 page 13 of 50 Artificial Neural Network Approach No need for normal distribution on input data Flexibility on input data format Improved classification accuracy Robust and reliability

14 page 14 of 50 ………………………… …………………… Artificial Neural Network Is Defined by... Processing elements Organized topological structure Learning rules

15 page 15 of 50 Processing Element (PE)  f(x) Output Input PE Artificial counterparts of neurons in a brain W j1 W j2 W j4 W j3 W j5

16 page 16 of 50 PE’s Output Function of Processing Elements ………………………… …………………… unit j o1o1 o2o2 o3o3 f w j1 w j2 w j3 ojoj PE’s Inputs Receive outputs from each PEs locate in previous layer. Compute the output with a Sigmoid activation function F( Sumof( O i * W ji) ) Transfer the output to all the PEs in next layer

17 page 17 of 50 Input layerHidden layerOutput layer ………………………… …………………… Input vector i(x 1, x 2, … x n ) Output vector i(o 1, o 2, … o m ) Artificial Neural Network Architecture - Backpropagation ANN (BPANN) Feed InformationAnalysis Result

18 page 18 of 50 Pattern Recognition Vegetation: (10, 89) > (1,0,0) (11,70) > (1,0,0) … … … (1,0,0) Water: (10, 21) > (0,1,0) (15, 32) > (0,1,0) … … … (0,1,0) Soil: (50, 40) > (0,0,1) (52, 40) > (0,0,1) … … … (0,0,1) Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Wa t er Soil

19 page 19 of 50 ANN Design Pattern Input layerHidden layerOutput layer …………………… How many PEs we need - Basic rules in designing an ANN. Input layer PEs - by dimension of input vector Output layer PEs - by total number of patterns (classes) …………………… Feed Forward Back-Propagate (0 ~ 255) (0 ~ 1) Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Water Soil

20 page 20 of 50 ANN Training - From Pattern to Land Cover Category Pattern …………………… …………………… Feed Forward Back-Propagate Vegetation: (10, 89) > (1,0,0) (11,70) > (1,0,0) … … … (1,0,0) Water: (10, 21) > (0,1,0) (15, 32) > (0,1,0) … … … (0,1,0) Soil: (50, 40) > (0,0,1) (52, 40) > (0,0,1) … … … (0,0,1) Land Cover Category (Vegetation) 0 (Water) 0 (Soil) A vegetation pixel (10, 89) > (1,0,0)

21 page 21 of 50 After Training water Vegetation Soil …………………… …………………… x y 1 (Vegetation) 0 (Water) 0 (Soil) A new pixel (x,y), x in band 1, y in band 2 A Well-trained ANN

22 page 22 of 50 Problems with Traditional Back- Propagation ANN Approaches Time-consuming – Always over iteration, over 5 hours for a small case. Black box - uncontrollable training – Easily trapped in local minimum. Training result unpredictable

23 page 23 of 50 Optimization Techniques to ANN Approach – Data Representation - Transform to Binary and Gray code format. – Improved Convergence Algorithms - Apply Conjugate Gradient and Resilient Propagation. – Weight initialization - Linear Regression.

24 page 24 of 50 Data Representation Data representation - Use Binary format and gray code format instead of integer value. Avoid to compute in the saturation range of the activation function Increased the computation space (from 6 to 48) Make the value more smoothly - Gray Code

25 page 25 of 50 Saturation Range of Activation Function - Sigmoid Function Saturation Original Inputs: 0 ~ 255 New Format: 0 or 1

26 page 26 of 50 Coding Method Integer format Binary format Forest Pixel 6 Integers48 Integers

27 page 27 of 50 Gray Code vs. Binary Format Gray Code is another coding system, similar to binary Can better represent continuous value Binary formatGray code formatInteger

28 page 28 of 50 Improved Convergence algorithms To Make the convergence more robust and reliable, apply: Conjugate Gradient (CG) Resilient Propagation (RPROP)

29 page 29 of 50 Error Space and Weight Adaptive Total Error Steps Wij Wkl

30 page 30 of 50 Weight Adaptive with Conjugate Gradient Wij Error Global minimum

31 page 31 of 50 Resilient Propagation (RPROP)

32 page 32 of 50 Weight initialization Cut convergence time, do weight initialization using linear regression to pre-process the internal weight, make ANN adapted to the target pattern before training.

33 page 33 of 50 Data and Experiments Study Area Data Distribution Fine tuning the ANN structure Classification Result

34 page 34 of 50 Study Area - Rhode Island 1999 Landsat-7 Enhanced Thematic Mapper Plus (ETM+) Image Band 4,3,2 In RGB Band 5,4,3 In RGB

35 page 35 of 50 Training and Testing Pattern

36 page 36 of 50 Distribution of Training Data Patterns Plotted by band 3 x 4 Patterns Plotted by band 4 x 5 Deciduous forest Turf / Grass Barren land Conifer forest Forest Wetland Mixed forest Brush Land Non-Forest Wetland Water Urban area

37 page 37 of 50 ANN Design and Tuning Number of PEs in input layer = Number of spectral Bands in the remote sensing image Number of PEs in output layer = Number of land cover categories Number of PEs in hidden layer(s) to be determined for best performance

38 page 38 of 50 ANN Design and Tuning - Before And After Optimization

39 page 39 of 50 Classification Result

40 page 40 of 50 Classification Result - A Close Look Rhode Island 1999 ETM+ Rhode Island 1999 Land-use and Land-cover map

41 page 41 of 50 Accuracy Assessment

42 page 42 of 50 Discussion and Conclusion Accuracy Gray code Vs. Integer format Robust and reliability Discuss on number of hidden layer PEs

43 page 43 of 50 Accuracy Comparison Non-optimized ANN Accuracy % Optimized ANN Accuracy %

44 page 44 of 50 Gray-code vs. Integer Value Comparison Between Gray Code and Integer Format

45 page 45 of 50 Resilient Propagation and Weight Initialization Improvement in robust and reliability

46 page 46 of 50 Hidden Layer PE Number’s Influence On Classifier’s Performance ( 48 inputs, 150 hidden neurons, 10 output classes) ( 48 inputs, 250 hidden neurons, 10 output classes) ( 48 inputs, 350 hidden neurons, 10 output classes)

47 page 47 of 50 Hidden Layer PE Number’s Influence On Classifier’s Performance. 350 hidden layer PEs provide the best performance

48 page 48 of 50 Conclusion Coding method, especially the Gray code is important to Remote Sensing image classification. Optimized ANN provide a robust and reliable classification solution Optimized ANN lead to higher classification accuracy Improved natural resources mapping

49 page 49 of 50 Acknowledgement The research was funded by NASA (Grant No. NAG5-8829) to Dr. Y.Q. Wang Dr. Y.Q. Wang (major professor) Dr. Pete August (NRS, committee member) Dr. Ken Yang (EE&CE) Dr. Tom Boving (GEO) Lab for Terrestrial Remote Sensing Environmental Data Center

50 page 50 of 50 Thank You!


Download ppt "Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035."

Similar presentations


Ads by Google