1. Vegetation Plot Management: A National Plots Database Demo Funding: National Science Foundation (DBI-9906838) John Harris - NCEAS Robert K. Peet - University of North Carolina

2. Agenda: Over-Arching Concepts Project Overview · Impact · Database Design · System Architecture · Challenges Use-Case Example: Wisconsin Data Data Management Recommendations Future Directions

3. The US National Plots Database Project supported by: National Center for Ecological Analysis & Synthesis U.S. National Science Foundation USGS-BRD Gap Analysis Program ABI / The Nature Conservancy Project organized and conducted by: Robert K. Peet, University of North Carolina Marilyn Walker, USDA Forest Service & U. Alaska Dennis Grossman, The Nature Conservancy / ABI Michael Jennings, USGS-BRD & UCSB John Harris, NCEAS

4. Collection Integration Archival Extraction Analysis Publication

5. Development Cycle Preliminary design Build prototype #1 interface User evaluates interface Evaluation studied by designer Design modification are made Build prototype #n interface Supported by 3 other NCEAS Developers Database Design: Aug. 2000 – Jan. 2001 Interface Design: Nov. 2000 – Feb. 2001 Backend Development: Jan. 2001 - Interface Development: Mar. 2001 – Backend Version: Prototype 3 Interface Version: Prototype 1 Expected Beta Release: Late Sept. – Mid Oct.

6. Plots Database Design

7.  Support the National Vegetation Classification.  Provide a comprehensive facility to store the most commonly collected vegetation plot data attributes.  Provide the user with a large number of user-defined attributes to store not-so-commonly collected data.  Integrate plots with the dynamic plant taxonomy and vegetation community data. Design meeting September 2000, NCEAS Plots Database Design Goals

14. Taxonomy Module Smithsonian meeting: Peet-Taswell model vs Berendsohn model FGDC Biological Nomenclature Working Group Update on ABI & HDMS Prospects for implementation The difficult choice

15.  Logical separation of a "taxonomic name" with the "taxonomic concept", so that taxonomic data can be stored at the most 'atomic' level without ambiguity  The ability to incorporate multiple organizations' 'views' of how a taxonomic name is applied to a taxonomic concept  The ability to link a taxonomic name used in the Plots database with a 'name - concept' pair in the taxonomic database. *Although one can store vegetation community data in the same database table-structure as the plant taxonomy database, we have implemented two separate table structures and have created two separate data sets. Taxonomy Database Design Goals

18. Representative tools reflect the desire to have the following features: High performance Robust Open architecture Platform neutral Scaleable Development Choices

19.  Java -- Write once, Cross platform – Linux, Windows, MacOS*  Java Servlet -- Dynamic, database-driven, web content  JDBC -- Connect to any database - Oracle, PostgreSql, SQL Server backend  Swing -- Classy interface tools  Beans -- Reusable components Features - JAVA * Not tested yet :-)

20. XML: is the format for structured data on the Web. Simple and flexible data conversions, using XSLT Straightforward to write generic tools which export parts of a relational database as XML encoded data, or even to write generic code that serializes Java (or other) objects as XML data structures. Examples later… Features - XML

21. An Example Workflow Using Wisconsin Plots Data What data integration means to us Taxonomic / Semantic Integration Data formatting for database ingestion General Comments about Current Format Data Parsing Transformation to XML standard Legacy Data Loader

22. Plots Data Integration & DB Ingestion Reformat by Hand Research MS Access MS Excel Perl Shell scripts ? Integration What is meant by data integration? … Plots DB

23. Taxonomic Integration Carya ovata (Miller)K. Koch Carya carolinae-sept. (Ashe) Engler & Graebner Carya ovata (Miller)K. Koch sec. Gleason 1952 sec. Radford et al. 1968 Splitting one species into two illustrates the ambiguity often associated with scientific names. If you encounter the name “Carya ovata (Miller) K. Koch” in a database, you cannot be sure which of two meanings applies. Integration

24. Semantic Integration of Plot Attributes ‘Basic yet Important’ Integration Cover Scales Strata Dimensions Environmental Attributes

25. Integration Parse Data from Forms into Table Structure to be Transformed into XML Consistent with the Database Structure Text Forms Columnar Tables XML

26. Integration Parsed Data Text Forms Columnar Tables

27. Integration Transform Parsed Data to XML Consistent with the Plots Database Columnar Tables Plots DB XML Legacy Data Loader Data Definition (XML)

28. Integration Data Definition (XML) – Single file siteData.csv ’,’ site data plotCode authorPlotCode 1 communityName 2 …

29. Integration Data Definition (XML) – Multiple files vegData.csv siteData.csv site data authorPlotCode '+' ‘,’ species plotName authorPlotCode 1 scientificName taxonName 2

30. Integration Plots Database XML

31. Existing Prototype Functionality

32. Vegetation Database Client

38. Vegetation Desktop Database Client

43. Extra slides to follow:

46. General Data Management Practices general formats weird formats unusable formats modeled the software after the way that people collect plots data -- at least that is what I thought At times tortuous path to the database in terms of reformating class indicies (these are rectified at the plots loading software step)

47. Management Case: Example from Wisconsin  Baraboo Hills -- Collected Yesterday  PEL -- Legacy Data

48. Data Transformation of Forms