2006 ESRI International Users ConferenceAugust 8, 2006 Spatial Data Infrastructure and Data Preservation in North Carolina Jefferson F. Essic, Robert Farrell, Steven P. Morris, and James Tuttle North Carolina State University Libraries

2 Overview Spatial Data Preservation: Values and Considerations NC Geospatial Data Archiving Project Approaches to Preservation Challenges Workflow NC Spatial Data Infrastructure NC OneMap Regional/Local Partnerships and Data Sharing Coordinated Content Transfer Historic and Geologic Map Preservation Project

3 Today’s geospatial data as tomorrow’s cultural heritage Future uses of data are difficult to anticipate (as with Sanborn Maps).

4 Temporal Data Supports Decision Making Land use change analysis Real Estate trend analysis Site selection (past uses?) Forecasting Parcel Boundary Changes North Raleigh, NC

5 Time series – Ortho imagery Vicinity of Raleigh-Durham International Airport

6 Geospatial Data: Risks Producer focus on current data Future support of data formats in question Shift to web services- and API-based access Inadequate or nonexistent metadata Increasing use of spatial databases for data management Many digital archiving challenges …

7 Geospatial data types: Aerial imagery 85+ NC counties with orthophotos 1-5 flights per county gb per flight

8 Geospatial data types: Vector & Tabular Economic, infrastructure, and ethnographic data

9 Geospatial data types: Cartographic Project Files Counterpart to the map is not just the dataset but also models, symbolization, classification, annotation, etc.

10 How would you describe your current geospatial archive? Last week’s set of nightly tape backups Several boxes of CD’s and DVD’s Bob’s hard drive A collection of files in our “GIS Folder” A stand-alone spatial database The data back-end for our internet mapping application An enterprise GIS

11 NC Geospatial Data Archiving Project (NCGDAP) Partnership between university library (NCSU) and state agency (NCCGIA), with Library of Congress under the National Digital Information Infrastructure and Preservation Program (NDIIPP) One of 8 initial NDIIPP partnerships Focus on state and local geospatial content in North Carolina (state demonstration) Tied to NC OneMap initiative, which provides for seamless access to data, metadata, and inventories Objective: engage existing state/federal geospatial data infrastructures in preservation Serve as catalyst for discussion within industry

12 Different Ways to Approach Preservation Technical solutions: How do we archive acquired content over the long term? Tools Hardware Software Cultural/Organizational solutions: How do we make the data more preservable—and more prone to be archived—from point of production? Collaboration Education Feedback

13 Technical Approaches Receive data as is – variety of distribution methods Migration of some at-risk formats Metadata remediation, standardization, and synchronization Distilling complex objects into repository ingest items (not easy) Using DSpace for demonstration purposes In the development: use METS record as dormant item “brain” within the repository Some unsustainable activities – for learning experience

14 Cultural/Organizational Approaches Feedback to metadata outreach program Feedback to coordinating bodies on adherence to content standards Engage existing spatial data infrastructure in archiving and preservation Engage software vendors and standards community Cross-fertilize with other national archiving efforts Current use and data sharing requirements – not archiving needs – drive improved preservability of content and improvement of metadata

15 Challenge: Vector Data Formats No widely-supported, open vector formats for geospatial data Spatial Data Transfer Standard (SDTS) not widely supported Geography Markup Language (GML) – diversity of application schemas and profiles threatens permanent access Spatial Databases The sum is more than the whole of the parts, and the sum is very difficult to preserve Can export individual data layers for curation Some thinking of using the spatial database as the primary archival platform

16 Challenge: Geospatial Web Services How to capture records from decision- making processes? Possible: Atlas collections from automated image capture Web 2.0 impact: Emerging tiling and caching schemes (archive target?)

17 General Workflow 1)Receive Data from Agency 2)Copy data from agency source to NCSU workstation 3)Create Dspace collection “space” for the data 4)Create administrative metadata 5)Process geospatial metadata 6)Scan geospatial formats and migrate to archival format 7)Ingest original and archival data objects, and geospatial administrative metadata to Dspace

18 NCGDAP Leveraging Existing SDI NC OneMap: "Historic and temporal data will be maintained and available,” RAMONA Metadata outreach and content standards Regional Partnerships WGRT and other Coordination Efforts Data Sharing Agreements Frequent communication and discussion among geospatial data community

19 NC Spatial Data Infrastructure: NCOneMap NC OneMap is a next generation mechanism to coordinate and disseminate geographic information in North Carolina and interact with the NSDI. Objectives: Build a common understanding of North Carolina data resources Enable widespread access and distribution of geospatial data

20 NC OneMap Viewer

21 NC OneMap Objectives (cont.): Develop ongoing data inventory for all geospatial data holdings RAMONA – Develop content standards for key data themes NC Geographic Information Coordinating Council (GICC) One of the defined characteristics of NC OneMap is that “Historic and temporal data will be maintained and available”.

22 Emerging Regional Partnerships Focused on development of shared infrastructure for cultivating access to data Becoming test beds for innovation in the area of data sharing and data management, including archiving

23 Local Govt. Data Sharing Becoming more open, fewer agreements to sign Recent survey: over 20 state and federal agencies use local data Problem of local governments being swamped by requests Many requests are more compelling than “archiving” Content transfer is non-trivial – large dataset sizes, small rural staffs, technical limitations

24 Coordinated Content Transfer Will allow one data snapshot to be accessible by multiple agencies Question: Capture frequency of data snapshot? Survey in-the-works to identify local government best practices, consumer agencies needs Working Group for Roads and Transportation (WGRT) Stakeholder group working to build data depository for statewide local road data First serious effort to develop a plan for local-to-state data sharing on a regular basis

25 Preservation of Digital Geologic and Historic Maps Georeferenced over 450 maps scanned by NC Geologic Survey Maps are available for download at 15-min topo maps 1:500,000 – 1:2.5 M 1:31,680 – 1:430,000 1,200 – 24,000

26 Questions? Jeff EssicSteve Morris Data Services LibrarianHead, Digital Library InitiativesNCSU Libraries Ph: (919) Ph: (919)