1. Methods of Validating Maps of Deforestation and Selective Logging Carlos Souza Jr. souzajr@imazon.org.br Instituto do Homem e Meio Ambiente da Amazônia—Imazon Caixa Postal 5101, Belém, PA, Brasil  CEP 66613-397 Department of Geography – University of California Santa Barbara, California 93106, USA [becky,matzke,carlos,mclark,numata,dar]@geog.ucsb.edu

2. Workshop Objectives to propose protocols for ground-based accuracy assessments of remotely sensed estimates of deforestation and selective logged forests. to address means of establishing error bars and accuracy estimates pertaining to deforestation and selective logging products.

3. Protocol for Accuracy Assessment of RS Products Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment Stephen V. Stehman Raymond L. Czaplewski RSE, 1998. 64:331-344

4. Sampling Protocol Sample unit  is a location (point) or an area (pixel or polygon) used to assess the accuracy of a map.  links the map and the reference data. Pixel sampling unit Reference data (Ikonos) Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment Stephen V. Stehman Raymond L. Czaplewski RSE, 1998. 64:331-344

5. Sampling Protocol Sampling method  Requires an unbiased sample Simple random Stratified random Cluster  Primary (psu)  Secondary (ssu) Polygon psu ssu 5x5 pixel block ssu psu “A scientifically defensible accuracy assessment requires a probability sampling” Stephen V. Stehman Raymond L. Czaplewski RSE, 1998. 64:331-344 Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment

6. Response Design It is a classification scheme applied to the sampling unit.  Labeling protocol Assigns a land-cover classification to the sampling unit based on the evaluation protocol:  Land cover proportion  Predominant land-cover  Centroid  Multiple land cover classification  Fuzzy classification Evaluation: 50% vegetation 40% bare soil 10% shrubs Labeling: Pasture Evaluation: pixel center Labeling: Forest Stephen V. Stehman Raymond L. Czaplewski RSE, 1998. 64:331-344 Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment

7. Reference Data D – Decidious C – Conifer AG – Agriculture SB - Shrub 43414111510375 104 90374 115 1985110 100 85816 1152422465 D C AG SB Row Total D C AG SB Column Total Overall Accuracy = (65+81+85+90)/434 = 74% Classified Data D = 65/75 = 87% C = 81/103 = 79% AG = 85/115 = 74% SB = 90/141 = 64% Producers Accuracy D = 65/115 = 57% C = 81/100 = 81% AG = 85/115 = 74% SB = 90/104 = 87% Users Accuracy Source: Congalton and Green (1999) Legend Stephen V. Stehman Raymond L. Czaplewski RSE, 1998. 64:331-344 Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment Estimation and Analysis

8. Problems with the Standard Protocol

9. Study Area

10. Sampling Design  Flight-line randomly sampled by time-code  Cluster samples centered on flight-line  Reference sample unit = 30 m x 30 m area  Five reference samples recorded per mosaic

11. Response Design

12.  Labeling protocol: dominant land-cover 34 1 2 0 Evaluation: 100% Pasture Label: “Pasture” Evaluation: 100% Secondary Forest Label: “Secondary Forest” Evaluation: 55% Pasture 45% Secondary Forest Label: “Pasture” Evaluation: 100% Primary Forest Label: “Primary Forest”

13. Accuracy Assessment Methods and Analysis Correct geolocation errors between video mosaics and map Randomly select image frames from video based on time code Mosaic and georectify video and overlay cluster samples Group (i.e. five interpreters) re- interprets video to generate ‘gold’ reference data set Five individuals independently interpret video to produce five reference data sets Remove mixed samples and edge pixels; identify change Parameters estimated from error matrices: overall accuracy, producer’s accuracy (omission errors), user’s accuracy (commission errors), Kappa (KHAT) statistics, Kappa variance (for statistical comparisons) Interpreter vs. interpreter accuracy Interpreter vs. map accuracy ‘ Gold’ reference vs. map accuracy

14. Influence of reference data corrections on map accuracy *statistically different Kappa coefficients (95% confidence level)

15. Area estimation accuracy assessment 33% 21% 23% 17% Average = 23.50% Stdev = 6.81%

16. Deforestation Estimates in Acre Area of Forest Fragments: 2864.9 km 2

17. Are the Fragments “Capoeiras”? Deforestation Age Forest Deforestation until 2000 Water Forest – Prodes digital Landsat Image Classification (Roberts et al., 2003) Fragments Classification (INPE - Prodes Digital)

18. Scale Effect on Mapping Deforestation 1:50.000 1:250.000 9 km

19. 1:50.000 1:250.000 Scale Effect on Mapping Deforestation

20. Deforestation by Classes of Size Contribuição p/ o Desmatamento (%) Contribuição p/ o Desmatamento (%) Número de Polígonos

21. Classification Errors in 2002 Old deforestation Forest Increment Incremento 2002 - INPE Classification Imazon/ImacClassification - Prodes

22. Accuracy estimates of deforestation and selective logging products Reference Data  High resolution satellite image Ikonos  Field data Several sources

23. Ikonos Data for LBA-Ecology Study Sites Hurtt et al., (2003)

24. Proposal Map accuracy  Sample design: random cluster sampling  Response design: ?  Estimation design: ?  Correct reference data errors  Several research groups interpreting the reference data Include area estimation accuracy