1. Jason Roberts, Ben Best, Dan Dunn, Eric Treml, Pat Halpin Duke University Marine Geospatial Ecology Lab 4-Mar-2009

2. Talk outline Overview of MGET Quick tour of the MGET tool collection Example application: habitat modeling

3. What is MGET? A collection of geoprocessing tools for marine ecology Oceanographic data management and analysis Habitat modeling, connectivity modeling, statistics Highly modular; designed to be used in many scenarios Emphasis on batch processing and interoperability Free, open source software Written in Python, R, MATLAB, C#, and C++ Minimum requirements: Win XP, Python 2.4 ArcGIS 9.1 or later currently needed for many tools ArcGIS and Windows are only non-free requirements Also useful for terrestrial problems!

4. Set goals & priorities Make & implement EBM decisions Monitor & assess Develop conceptual models Analyze data & develop models and scenarios Collect physical, biological, and socioeconomic data Visualize scenarios

5. Set goals & priorities Make & implement EBM decisions Monitor & assess Develop conceptual models Analyze data & develop models and scenarios Collect physical, biological, and socioeconomic data Visualize scenarios Monitoring & assessment tools Project management tools Data collection and management tools Conceptual modeling tools Data processing tools Modeling tools -Model development tools -Watershed models -Dispersal and habitat models -Marine ecosystem models -Social science models Sector-specific decision support tools -Conservation and restoration site selection -Coastal zone management tools -Fisheries management tools -Hazard assessment and resiliency planning tools -Land use planning tools Stakeholder communication & engagement tools Scenario visualization tools

6. Set goals & priorities Make & implement EBM decisions Monitor & assess Develop conceptual models Analyze data & develop models and scenarios Collect physical, biological, and socioeconomic data Visualize scenarios Monitoring & assessment tools Project management tools Conceptual modeling tools Sector-specific decision support tools -Conservation and restoration site selection -Coastal zone management tools -Fisheries management tools -Hazard assessment and resiliency planning tools -Land use planning tools Stakeholder communication & engagement tools Scenario visualization tools Data collection and management tools Data processing tools Modeling tools -Model development tools -Watershed models -Dispersal and habitat models -Marine ecosystem models -Social science models Data collection and management tools Data processing tools Modeling tools -Model development tools -Watershed models -Dispersal and habitat models -Marine ecosystem models -Social science models MGET

7. MGET’s software architecture MGET “tools” are really just Python functions, e.g.: MGET exposes them to several types of external callers: def MyTool(input1, input2, input3, output1)

8. MGET interface in ArcGIS The MGET toolbox appears in the ArcToolbox window

9. MGET interface in ArcGIS Drill into the toolbox to find the tools Double-click tools to execute directly, or drag to geoprocessing models to create a workflow

10. Integration The Python functions can invoke C++, MATLAB, R, ArcGIS, and COM classes.

11. MGET utilizes a lot of other software Interpreters / Runtimes Python MATLAB Component Runtime R R Packages gam MASS mgcv rgdal ROCR Applications ArcGIS NOAA CoastWatch Utilities Python Packages docutils httplib2 lxml netcdf4 numpy osgeo pydap pyparsing pyproj pywin32 rpy setuptools C Libraries GDAL/OGR gzip hdf libxml libxslt netcdf proj4 zlib All but one of these (pywin32) are installed automatically

12. Quick tour of the tools

13. Analyzing larval connectivity Coral reef ID and % cover maps Ocean currents data Tool downloads data for the region and dates you specify Larval density time series rasters Edge list feature class representing dispersal network Original research by Eric A. Treml

14. Converting data

15. Batch processing Copy one raster at a time

16. Batch processing Copy rasters that you list in a table

17. Batch processing Copy rasters from a directory tree

18. Tools for specific products Downloads sea surface height data from http://opendap.aviso.oceanobs.com/thredds

19. Identifying SST fronts ~120 km AVHRR Daytime SST 03-Jan-2005 28.0 °C 25.8 °C Mexico Front Cayula and Cornillion (1992) edge detection algorithm Frequency Temperature Optimal break 27.0 °C Strong cohesion  front present Step 1: Histogram analysis Step 2: Spatial cohesion test Weak cohesion  no front Bimodal Example output Mexico ArcGIS model

20. Identifying geostrophic eddies Aviso DT-MSLA 27-Jan-1993 Red: Anticyclonic Blue: Cyclonic Negative W at eddy core SSH anomaly Available in MGET 0.8 Example output

21. Mapping species biodiversity

22. Invoking R from ArcGIS

23. Example application: habitat modeling Chlorophyll SST Bathymetry Presence/absence observations Sampled environmental data Multivariate statistical model Probability of occurrence predicted from environmental covariates Binary classification Warning: Habitat modeling is complicated! This simplified example is meant to briefly illustrate tools. Consult the literature for best practices!

24. Focal species: Stenella frontalis Photo: Garth Mix Map: OBIS-SEAMAP Common name: Atlantic Spotted Dolphin Distribution: Tropical and warm temperate Atlantic Study area: Eastern U.S.

25. Species observation data The Ocean Biogeographic Information System (OBIS) is a global database of marine species observations. The OBIS-SEAMAP system at Duke University holds the records for seabirds, marine mammals, and sea turtles, including records gathered during NOAA cruises.

26. Environmental predictor variables Bathymetry: ETOPO2V2 from NOAA NGDC SST: Monthly climatological 4km AVHRR Pathfinder from NOAA NODC Chlorophyll: Monthly climatological SeaWiFS chlorophyll-a from NASA GSFC Images shown above are for month of March

27. Step 1: Download species points Download points using MGET tool: Presence: Records of Stenella frontalis Absence: Records of other cetaceans The tool uses the DiGIR protocol to retrieve data from OBIS servers

28. Red: Presence Green: Absence

29. Step 2: Convert oceanography to Arc rasters 1. Download with FTP from NOAA and NASA: ETOPO2 bathymetry – 1 binary file AVHRR Pathfinder monthly climatological SST – 12 HDF files SeaWiFS monthly climatological chlorophyll – 12 HDF files 2. Convert to ArcGIS rasters using MGET tools:

30. Step 3: Sample oceanography at points Need to sample rasters and populate fields Must sample SST and chlorophyll by date

31. Step 3: Sample oceanography at points Sampling bathymetry is easy because it is static To sample dynamic data such as SST and chlorophyll, you must first calculate the paths to rasters to sample from the points’ dates Then use an MGET batch sampling tool

32. Step 4: Create exploratory plots Best predictors: SST and Chl

33. Step 5: Fit, evaluate, and predict model Presence ~ s(SST) + s(log10(Chlorophyll))

34. Partial plots produced by the Fit GAM tool SST log10(Chlorophyll) s(SST,8.97) s(log10(Chlorophyll),5.6) Presence more likely at higher SSTPresence more likely at lower Chl

35. Plotting a receiver operating characteristic curve

36. The ROC plot False positive rate True positive rate Cutoff = 0.020 By default, tool selects the cutoff closest to the point of perfect classification (0, 1) Model summary statistics: Area under the ROC curve (auc) = 0.960779 Mean cross-entropy (mxe) = 0.030566 Precision-recall break-even point (prbe) = 0.001866 Root-mean square error (rmse) = 0.087781 Contingency table for cutoff = 0.019638: Actual P Actual N Total Predicted P 287 3541 3828 Predicted N 26 32408 32434 Total 313 35949 36262 Accuracy (acc) = 0.901633 Error rate (err) = 0.098367 Rate of positive predictions (rpp) = 0.105565 Rate of negative predictions (rnp) = 0.894435 True positive rate (tpr, or sensitivity) = 0.916933 False positive rate (fpr, or fallout) = 0.098501 True negative rate (tnr, or specificity) = 0.901499 False negative rate (fnr, or miss) = 0.083067 Positive prediction value (ppv, or precision) = 0.074974 Negative prediction value (npv) = 0.999198 Prediction-conditioned fallout (pcfall) = 0.925026 Prediction-conditioned miss (pcmiss) = 0.000802 Matthews correlation coefficient (mcc) = 0.246384 Odds ratio (odds) = 101.026394 SAR = 0.650065 ROC summary stats for cutoff:

37. Predicting presence for oceanographic rasters

38. Rasters output by the Predict GAM tool Predicted presence: Range: 0 - 0.25 Standard errors: Range: 0 - 0.11 Binary classification: Species range map produced by classifying presence into 0 or 1 according to ROC cutoff Similar to OBIS-SEAMAP range map? Predictions for October

39. Acknowledgements A special thanks to the many developers of the open source software that MGET is built upon, including: Guido van Rossum and his many collaborators; Mark Hammond; Travis Oliphant and his collaborators; Walter Moreira and Gregory Warnes; Peter Hollemans; David Ullman, Jean-Francois Cayula, and Peter Cornillon; Stephanie Henson; Tobias Sing, Oliver Sander, Niko Beerenwinkel, and Thomas Lengauer; Frank Warmerdam and his collaborators, Howard Butler; Timothy H. Keitt, Roger Bivand, Edzer Pebesma, and Barry Rowlingson; Gerald Evenden; Jeff Whitaker; Roberto De Almeida and his collaborators; Joe Gregorio; David Goodger and his collaborators; Daniel Veillard and his collaborators; Stefan Behnel, Martijn Faassen, and their collaborators; Paul McGuire and his collaborators; Phillip Eby, Bob Ippolito, and their collaborators; Jean-loup Gailly and Mark Adler; the developers of netCDF; the developers of HDF Thanks to our funders:

40. For more information Download MGET: http://code.env.duke.edu/projects/mget Email us: jason.roberts@duke.edu, bbest@duke.edu Learn more about habitat modeling: Guisan, A., Zimmermann, N.E. (2000) Predictive habitat distribution models in ecology. Ecological Modelling 135, 147–186. Thanks for attending!