1. A methodology for translating positional error into measures of attribute error, and combining the two error sources Yohay Carmel 1, Curtis Flather 2 and Denis Dean 3 1 The Thechnion, Haifa, Israel 2 USDA, Forest Service 3 Colorado State University

2. Part 1: bridging the gap between positional error and classification error

3. classification error -- difference in pixel class between the map and a reference

4. Positional error (misregistration, location error) Is the gap between the true location of an item and its location on the map / image

5. Positional error may translate to thematic error positional error largely affects overall thematic error (often more than classification error) (Townshend et al 1992, Dai and Khorram 1998)

6. Positional error: RMSE = 2.51 m classification error: Accuracy Matrix: Goal 1: find a common denominator for both error types Goal 2: combine the two error types to get an overall estimate of error (in the context of temporal change) THE PROBABILITY THAT AN OBSERVED TRANSITION IS CORRECT

7. positional error affects thematic error

8. Expressing positional error in terms of thematic error Shift = 1, 0 Shift = 15, 7 Shift = 2, 3

9. Expressing positional error in terms of thematic error

10. Error model: step 1 (and step 2) of 5 RMSE Positional Error Classification Error

11. positional accuracy matrix A LOC classification accuracy matrix A CLASS Combined A BOTH A LOC 1, 1 *A CLASS 2, 1 / n +1 A LOC 2, 1 *A CLASS 2, 2 / n +2 A LOC 3, 1 *A CLASS 2, 3 / n +3 A BOTH 2,1 = + + Classified error model: step 3 (of 5) Combining the two accuracy matrices

12. Classified Reference Error model: step 3 Combined Error Matrix

13. Error model: step 4 Calculating the combined PCC, and the Combined user accuracy, p(C) Classified Reference PCC = 0.70 THE PROBABILITY THAT AN OBSERVED STATE IS CORRECT

14. Error model: step 5 Calculating multi-temporal indices One such index is The probability that an observed transition is correct The context of this model is temporal change. The goal is to provide indices for the reliability of an observed change

15. Example: vegetation changes in Hastings Nature Reserve, California

16. 1939 1956 1971 1995

17. Example: vegetation changes in Hastings, California RMSE 1939 = 3.53 m 1995 = 2.51 m User accuracy for: Grass in 1939 = 0.92 Trees in 1995 = 0.91 positional accuracy Classification accuracy 1939 1995 C1C1 C4C4

18. The probability that an observed transition is correct p(C 1 C 2 …C n ) = p(C 1 ) * p(C 2 ) * … * p(C n ) C1C1 C2C2 C3C3 C4C4 1939 1956 1971 1995

19. Error model: step 5 p(C 1 C 2 …C n ) = p(C 1 ) * p(C 2 ) * … * p(C n ) 1939 1956 1971 1995 C1C1 C2C2 C3C3 C4C4 The probability that an observed transition is correct: This probability may be calculated as the product of the respective user-accuracy value for the respective year and class

20. Example: vegetation changes in Hastings, California 1939 1995 GT 0.53 p(GRASS 1939 TREE 1995 ) = 0.53 0.22 p(G 1939 G 1956 G 1971 T 1995 ) = 0.22 1939 1956 1971 1995 GGGT

21. Transition type Nature of transition Proportion in 1939 Probability of being correct, given : positional error Classif. error Combined error GGGG Grassland does not change 0.430.280.690.21 CGGGChaparral burnt in 1955 fire 0.060.290.600.22 All 69 transitions involving forest 0.660.260.590.20 Averaged across the entire study area 10.290.670.22 Indices of accuracy of multi-temporal datasets

22. (1)errors in each time step are independent of errors in other time steps (2)positional and classification errors are independent of one another A simulation study was conducted in order to evaluate how robust is the model in general and in particular -- to violations of two assumptions:

23. Maps for simulations High autocorrelation, equal class proportions Low autocorrelation, unequal class proportions Int. J. Rem. Sens. 2004

24. Original map Spatial error Classification error Both error types Simulation

25. Model simulations were conducted under a range of values for: Number of map categories (2-4) Class Proportions in the original map Auto-correlation in the original map Auto-correlation in classification error Classification error rate Positional error rate Correlation in error structure between time steps Correlation between the two error types

26. Some results of simulation runs

27. When models assumptions are not met, model fit decreases Maximum correlation found in real datasets* *IEEE GRSL 2004 Maximum correlation found in real datasets* *IEEE GRSL 2004

28. Transition type Nature of transition Proportion in 1939 Probability of being correct, given : positional error Classif. error Combined error GGGG Grassland does not change 0.430.280.690.21 CGGGChaparral burnt in 1955 fire 0.060.290.600.22 All 69 transitions involving forest 0.660.260.590.20 Averaged across the entire study area 10.290.670.22 Indices of accuracy of multi-temporal datasets

29. PART 2: Controlling data-uncertainty via aggregation Pixel size = 0.3 m Grid cell size = 15 m = 2500 pixels

30. Map aggregation = image degradation overlay a grid of cells on the image (cell >>pixel) and define the larger cell as the basic unit Pixel size = 0.3 m Grid cell size = 15 m = 2500 pixels

31. ‘soft’ aggregation ‘hard’ aggregation Map aggregation = image degradation overlay a grid of cells on the image (cell >>pixel) and define the larger cell as the basic unit

32. Impact of positional error is largely reduced when cells are aggregated a b c

33. At the pixel level: only 55% of the pixels remained unaffected by a minor shift At the grid cell level: Impact of positional error is largely reduced when cells are aggregated

34. This trade-off calls for a model that quantifies the process to aid decisions on optimal level of aggregation Aggregation: Gain in accuracyBUTloss of information

35. A geometric approach to the impact of positional error

36. Effective positional error at the grid cell level  is the proportion of pixels that transgress into neighboring cells (RMSE units)

37. Positional error at the GRID CELL level p(loc) is the probability that positional error translates into attribute error p A (loc) is the same probability – in the context of a larger grid cell

38. The impact of aggregation on thematic accuracy 0.23 A p(loc) cell size error 0.6 m0.23 6 m0.14 60 m0.01

39. Conclusions positional error has a large impact on thematic accuracy, particularly in the context of change But can be easily mitigated: increase MMU to >10X[positional error] and do not worry about it. Within overall thematic error at the pixel level – classification error component is typically smaller than the positional error component, but is more difficult to get rid of by aggregation.

40. TODA THANK YOU