1. A COMPARISON OF APPROACHES FOR VERIFYING SOUTHWEST REGIONAL GAP VERTEBRATE-HABITAT DISTRIBUTION MODELS J. Judson Wynne, Charles A. Drost and Kathryn A. Thomas USGS- Southwest Biological Science Center, Colorado Plateau Research Station, Flagstaff, Arizona

2. Southwest Regional Gap Data products: land cover, stewardship and wildlife-habitat maps 833 wildlife habitat maps Information to land managers, researchers, policy makers, and the general public AZ, CO, NM, NV, UT - Habitat correlates identified via literature reviews

3. Objectives Compare and contrast the two approaches. Review approach to accuracy assessment used in previous GAP programs. Describe an alternative approach to GAP accuracy assessment. Identify best approach.

4. Accuracy Assessment Agreement between the expected and observed Utility: - Evaluate model quality - Identify and correct for error - Compare techniques, algorithms, and model developers - Assess relevance in decision making

5. Without Accuracy Assessment… Models are untested hypotheses Does =+ ? = ? And does

6. Gap and “Accuracy Assessment” Species list approach - compiled list of species within given management areas - overlaid on predictive distribution maps - “measure of agreement” with omission and commission errors

7. Sample Size Arizona Accuracy Assessment GAP1 - Source data: USDA FS, NPS, AGFD, TNC 10Mammals 15Birds 10Reptiles 10Amphibians No. of AreasTaxonomic Group From Drost et al. (1999) 1. Intensive, area wide for ≥ one taxon - Criteria 2. Compilation of several surveys with ≥ one taxon 3. Compilation of secondary sources

8. Expansion Range Issues Require recent data to assess these issues Contraction

9. False Agreement Incidental/ accidental occurrences

10. False Agreement Observations not within habitat

11. Issues of Temporal Scale Time frame reflected in data

12. Issues of Spatial Scale For all groups, model accuracy <50% for areas <1000ha Accuracy increased as species list area increased 0.0120.56710Mammals <0.010.78715Birds 0.077 0.33910Reptiles 0.0260.48310Amphibians P-valueR2R2 No. of Areas Taxonomic Group Birds Arizona Accuracy Assessment GAP 1

13. Approach: Use of species occurrence data Measure of habitat rather than indirect measure of range More appropriate scale Statistically meaningful accuracy metrics A Higher Standard

14. - Presence only point data - Grid-based data - Presence/ absence point data Data Sources for Model Verification

15. Presence Only Data USDA Forest Service Arizona / New Mexico database - Point data - Museum specimen, trapping and other observations - Standardized format

16. Presence/ Absence Data AGFD Bat Data (Central Arizona) - Systematic sampling protocols - Seven year dataset - Multiple surveys/ year/ site - Point data

17. Grid Data Arizona Breeding Bird Atlas - Appropriate scale (~1 km 2 grid cell) - Rigorous sampling design - Recently completed - Statewide coverage - Sampling- multiple years/ site - Trained/ experienced observers

18. Systematic Typically random Data Collection Defined by sample design Variable Reliability Much larger Very small Sample Size Limited to comprehensive Poor Coverage Known for each dataset Generally not addressed Temporal Issues Close to map scale Very coarse Scale Verification DataSpecies List Comparison of Approaches

19. Conclusions Best approach: Occurrence/ verification data Verification data do exist Best available science Species lists inadequate for accuracy assessment