Methods of Validating Maps of Deforestation and Selective Logging Carlos Souza Jr. Instituto do Homem e Meio Ambiente da Amazônia—Imazon Caixa Postal 5101, Belém, PA, Brasil  CEP Department of Geography – University of California Santa Barbara, California 93106, USA

Workshop Objectives to propose protocols for ground-based accuracy assessments of remotely sensed estimates of deforestation and selective logged forests. to address means of establishing error bars and accuracy estimates pertaining to deforestation and selective logging products.

Protocol for Accuracy Assessment of RS Products Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment Stephen V. Stehman Raymond L. Czaplewski RSE, :

Sampling Protocol Sample unit  is a location (point) or an area (pixel or polygon) used to assess the accuracy of a map.  links the map and the reference data. Pixel sampling unit Reference data (Ikonos) Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment Stephen V. Stehman Raymond L. Czaplewski RSE, :

Sampling Protocol Sampling method  Requires an unbiased sample Simple random Stratified random Cluster  Primary (psu)  Secondary (ssu) Polygon psu ssu 5x5 pixel block ssu psu “A scientifically defensible accuracy assessment requires a probability sampling” Stephen V. Stehman Raymond L. Czaplewski RSE, : Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment

Response Design It is a classification scheme applied to the sampling unit.  Labeling protocol Assigns a land-cover classification to the sampling unit based on the evaluation protocol:  Land cover proportion  Predominant land-cover  Centroid  Multiple land cover classification  Fuzzy classification Evaluation: 50% vegetation 40% bare soil 10% shrubs Labeling: Pasture Evaluation: pixel center Labeling: Forest Stephen V. Stehman Raymond L. Czaplewski RSE, : Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment

Reference Data D – Decidious C – Conifer AG – Agriculture SB - Shrub D C AG SB Row Total D C AG SB Column Total Overall Accuracy = ( )/434 = 74% Classified Data D = 65/75 = 87% C = 81/103 = 79% AG = 85/115 = 74% SB = 90/141 = 64% Producers Accuracy D = 65/115 = 57% C = 81/100 = 81% AG = 85/115 = 74% SB = 90/104 = 87% Users Accuracy Source: Congalton and Green (1999) Legend Stephen V. Stehman Raymond L. Czaplewski RSE, : Response Design Protocol Sampling Design Protocol Estimation and Analysis Protocol Accuracy Assessment Estimation and Analysis

Problems with the Standard Protocol

Study Area

Sampling Design  Flight-line randomly sampled by time-code  Cluster samples centered on flight-line  Reference sample unit = 30 m x 30 m area  Five reference samples recorded per mosaic

Response Design

 Labeling protocol: dominant land-cover Evaluation: 100% Pasture Label: “Pasture” Evaluation: 100% Secondary Forest Label: “Secondary Forest” Evaluation: 55% Pasture 45% Secondary Forest Label: “Pasture” Evaluation: 100% Primary Forest Label: “Primary Forest”

Accuracy Assessment Methods and Analysis Correct geolocation errors between video mosaics and map Randomly select image frames from video based on time code Mosaic and georectify video and overlay cluster samples Group (i.e. five interpreters) re- interprets video to generate ‘gold’ reference data set Five individuals independently interpret video to produce five reference data sets Remove mixed samples and edge pixels; identify change Parameters estimated from error matrices: overall accuracy, producer’s accuracy (omission errors), user’s accuracy (commission errors), Kappa (KHAT) statistics, Kappa variance (for statistical comparisons) Interpreter vs. interpreter accuracy Interpreter vs. map accuracy ‘ Gold’ reference vs. map accuracy

Influence of reference data corrections on map accuracy *statistically different Kappa coefficients (95% confidence level)

Area estimation accuracy assessment 33% 21% 23% 17% Average = 23.50% Stdev = 6.81%

Deforestation Estimates in Acre Area of Forest Fragments: km 2

Are the Fragments “Capoeiras”? Deforestation Age Forest Deforestation until 2000 Water Forest – Prodes digital Landsat Image Classification (Roberts et al., 2003) Fragments Classification (INPE - Prodes Digital)

Scale Effect on Mapping Deforestation 1: : km

1: : Scale Effect on Mapping Deforestation

Deforestation by Classes of Size Contribuição p/ o Desmatamento (%) Contribuição p/ o Desmatamento (%) Número de Polígonos

Classification Errors in 2002 Old deforestation Forest Increment Incremento INPE Classification Imazon/ImacClassification - Prodes

Accuracy estimates of deforestation and selective logging products Reference Data  High resolution satellite image Ikonos  Field data Several sources

Ikonos Data for LBA-Ecology Study Sites Hurtt et al., (2003)

Proposal Map accuracy  Sample design: random cluster sampling  Response design: ?  Estimation design: ?  Correct reference data errors  Several research groups interpreting the reference data Include area estimation accuracy