U.S. Department of the Interior U.S. Geological Survey FY2010 Data Integration: Services, Capacity, Framework. September 28,

A Request for Funds  $500K in funding from FY2010 Bureau Shared Costs to complete short-term, targeted efforts that benefit USGS Science Data Integration. 2

Funding Request Outcomes  Deliver immediate benefit to solve an existing data integration challenge.  Create an infrastructure that can be leveraged.  Demonstrate a methodology and/or solutions architecture that can be repeated/replicated for other data.  Create an environment that allows future innovative applications to access USGS data. 3

Small Investment, Big Pay-Off One Use Case:  ~4,000 USGS scientists are licensed for ArcGIS  If ½ of those scientists save one pay period a year by reducing their manual efforts manipulating data  ~$8M of scientists’ time can be refocused on our science mission (based on $100K annual salary) 4

From a Scientist’s Perspective:  Data Integration is the ability to:  Discover the existence of data relevant to a study.  Access and evaluate that data.  Download and use data in an analytical framework.  Make the results available for others use.  These abilities are required by our scientists to do their job.  Enabling these abilities requires:  Services (Mostly Computer Services & Tools)  Capacity (Skills and Training)  Consistent framework (Solutions Architecture) 5

Council for Data Integration: On Services, Framework, Capacity  Build Web services and tools to discover, expose, and analyze data.  Build a framework to foster data management by providing solutions architecture, standards, data stewardship, data hosting, and publishing.  Build skills and capacity through training, education, workforce development, and cultural change. 6

The Council Approach:  Focus on targeted efforts that yield short- term benefits to the USGS science mission  Leverage existing capabilities within USGS  Leverage all USGS assets from bottom to top  Be sustainable –  By incorporating into existing efforts  By supporting a longer-term strategy 7

 Build upon existing efforts of science centers with targeted, tangible products  Empower scientists with data and tools  Seek substantial return on investment  Augment existing fundamental building blocks for data integration 8 The Council Approach:

The Council for Data Integration  Responsibilities: 1. Lead the development and implementation of the USGS data integration strategy 2. Provide recommendations for implementation of data integration guidelines 3. Promote Bureau-wide data integration  Membership: AD for Geospatial Information, Chief Scientists, key scientists and managers  Community of Practice Teams: Address technical issues 9

Council Members Betty Adrian Steve Aulenbach (NEON) Lee Allison (AZ) Leslie Armstrong Deb Baker Sky Bristol Frank D’Erchia Terry D’Erchia Jacque Fahsholtz Steve Fancy (NPS) Robert Fisher Dave Govoni Greg Gunther Lauren Hay Donn Holmes Harry House Richard Huffine Kate Kase Tim Kern Keith Kirk Karen Klima Linda Leake Fran Lightsom Bob Matthias Scott McEwen Mike McHale Mike Mulligan Pete Murdoch JC Nelson Callie Oblinger Bob Pierce Brian Reece Alyssa Rosemartin Carma San Juan Peter Schweitzer Sharon Shin Jenifer Sieverling Chris Skinner Mark Sogge Dave Soller Pat Stingley (DOI) Bob Swanson June Thormodsgard Roland Viger Brian Wee (NEON) Mingzhen Wei Jake Weltzin Sandy Williamson Kevin Gallagher (Co-Chair) Linda Gundersen (Co-Chair) Gladys Cotter Dave Ferderer Len Gaydos Marty Goldhaber Earl Greene Kevin Hope Bruce Jones Cheryl Morris Stan Ponce Roger Sayre Richard Signell Steve Tessler Community of Practice Council for Data Integration Members 10

CDI Accomplishments in 5 months  Chartered Council  5 Meetings (including 1 face-to-face)  Teams  Program Planning  Demonstration Project  Metadata  Data Publishing (initial focus on data.gov)  Wiki and document repository  High value opportunities  Draft 5-Year Program Plan 11

CDI – Wiki and Document Repository

Council for Data Integration Community met in Denver, September 15-17, 2009 Meeting Goals Build community by sharing stories about data integration efforts Identify high value data integration opportunities Agenda Highlights Climate Change: CEN, NCCWRC Ecosystems: GAM, WLCI, WERC, NPN Powell Center Data Blast: 20 posters and live demos Partners: AZ State Geologist, NPS, DOI, NEON Outcome Proposed six high value opportunity areas Council Face-To-Face 13

Over 25 High Value Opportunities Defined in 6 Overarching Themes Web Services Tools for Using Data Geospatial Availability Of Corporate Data Assets Architecture For Data Integration Metadata Registry & Data Publishing Cultural Changes and Workforce Planning … that will leverage resources to augment and extend existing science center data integration efforts to all USGS scientists 14

FY2010 Funding Request Details:  Make demonstrable progress towards implementing three high value areas:  Exposing “Corporate” Data  Organizing Science Models and Outputs  Preserving and Accessing Project Data while Training Scientists 15

USGS Corporate Data  User Story  As a scientist, I need a way to find, get and use USGS corporate data to explore new scientific discoveries and conclusions. I need a way to make sure I am getting the best available data and to access them in a way that I do not have to spend time transforming them into a format that I can use.  Challenge  USGS scientists (and many more external customers) manually download, reformat, and load data into ArcGIS and TNM. 16

Expose USGS Corporate Data - ( Services and Framework)  Approach  Develop web services to directly access USGS “corporate” data with ArcMap and TNM.  Benefit & Impact  Scientists save valuable time.  USGS “corporate” data is readily available via standard web services.  A framework/methodology for implementing web services is established which can be replicated. 17

Science Modeling Data  User Story  As a scientific modeler, I want a way to quickly access major modeling inputs (e.g., downscaled GCMs) in a form that will drive my modeling application. I also want a way to store, organize, and share the model outputs so that other scientists can access and use them and I can have a permanent online reference to these important artifacts.  Challenge:  USGS scientists need to share and publish project data and modeling outputs that do not currently have an established outlet. 18

 Approach  Leverage current hosting capabilities to serve as a repository for standardized modeling inputs and outputs. Use standard data cataloging techniques to document and expose best available data for discovery and access.  Benefit & Impact  Scientists’ time is saved by reducing data preparation time for established modeling/analysis workflows.  Model outputs are referenced, easily discovered, and accessed online.  Solutions architecture is portable to other efforts. 19 Modeling Hosting & Access Portal – ( Services and Framework)

Data Management & Upload Tools  User Story  As a USGS scientist, I have data that I have created or enhanced from other data, and I need a way to post this data online, document it so that others can understand it, share it with colleagues before it is published, and publish it electronically as a permanent data resource. I want to be able to integrate my project data with USGS corporate data and with other project data so that I can work with a broad range of data assets using the tools with which I am familiar.  Challenge  DOI and USGS need to compile, organize and manage data assets. Scientists and technology specialists need training and support to implement these tools. 20

 Approach  Create a database upload and registry tool for non-corporate databases. Provide training and support (e.g. swat team) to educate and assist.  Benefit & Impact  Project data that does not fit into a major USGS system will not be lost.  Project data that might not otherwise be accessible will be available to all scientists.  Scientists will receive valuable data management and tools training. 21 Data Management & Upload Tools – ( Services and Capacity)

Accountability: If funded the Council will -  Focus on clear deliverables and develop methodologies that can be replicated.  Track and report on progress.  Create a single account to track expenditures.  Redirect GIO resources to augment.  Leverage existing efforts in science centers. 22

Funding Request Outcomes  Deliver immediate benefit to solve an existing data integration challenge.  Create an infrastructure that can be leveraged.  Demonstrate a methodology and/or solutions architecture that can be repeated/replicated for other data.  Create an environment that allows future innovative applications to access USGS data. 23

A shared passion for … data integration 24