OneStop Project Update for WGISS 20 September 2017 Kenneth S. Casey, Ph.D. Deputy Director, NCEI Data Stewardship Division

OneStop Overview In response to the US federal Open Government Initiative and related policies, NOAA has committed to providing improved public access to all of its environmental information, to enable research and commercial innovation through ease of data discovery and use OneStop supports NOAA's efforts by leveraging existing access technologies and infusing specific innovations to provide improved discovery, access, and visualization services for NOAA’s data OneStop has had an initial focus on selected high-priority datasets archived at NESDIS National Centers for Environmental Information, but is being developed to eventually scale up to enable discovery to all of NOAA’s data OneStop is implementing the data discovery standards currently in use by WGISS and CWIC (CSW, OpenSearch). OneStop integrates capabilities of the Geoportal Server and ElasticSearch 5 to form the OneStop API

OneStop Approach Modernize high-priority data and metadata Update data and metadata to meet standards and best practices Copy data from tape to disk or cloud to support access Improve data infrastructure in a secure, user-centered way Improve rankings so most relevant data listed first (think Google) Improve “facets” to allow filtering of results (think Amazon) Support data visualization to enhance search experience (think Netflix) Improve performance and stability of interface to meet user expectations Enable user registration to support ratings, save searches, etc. Release software incrementally for internal stakeholder review and external testing

Yet Another Portal?? NO!! OneStop is improving not just discovery of, but also access to and usability of NOAA data Builds on foundation of existing, mature data standards and web services Emphasizes not just interface, but the supporting data infrastructure and metadata tools Supports users in connecting at whichever level they prefer

OneStop Data Framework30,000 ft Data Storage Services Catalog Services Showcase User Interface Data access, subset, visualization, and granule services Disk Storage Metadata Other User Interfaces Other Metadata Systems Other Data Access Systems Inside OneStop Outside OneStop BDP Cloud CLASS Tape For example, IDN and CWIC! BDP = Big Data Project

OneStop Metadata Framework Metadata editing interface is built on APIs, just as the Discovery UI

OneStop Key Milestones Estimated Completion Date Project Initiation Completed, Q4 FY15 Project Charter Completed, 10/1/2015 Architecture Design Review Completed, 1/4/2016 2 Data Groups Made OneStop-ready Completed, 6/30/2016 5 Data Groups Made OneStop-ready Completed, 9/30/2016 Software Release -- Beta Version Completed, 12/6/2016 Software Release -- Version 1.0 Completed, 4/18/2017 10 Data Groups Made OneStop-ready 3/31/2017 Software Release -- Version 1.1 Completed, 8/11/2017 Independent Evaluation Report 11/15/2017 15 Data Groups Made OneStop-ready 9/29/2017 Software Release -- Version 1.2 Software Release - Version 1.3 or 2.0 (CWIC Placed-in-Service date) 12/7/2017 9/10 complete and two more groups at > 85% readiness Questions around versioning resolved. Dec release number will depend on ElasticSearch 5 readiness Complete In Progress

OneStop Highlights OneStop data and metadata “readiness” criteria improved iteratively and now well established Nine data groupings consisting of hundreds of data collections now improved, progress continues on others (see details on next slide) OneStop central storage system operational and supporting movement of data from tape to disk for better access Releases 1 and 2 of Metadata Tools successfully demonstrated with initial import, export, and validate functions, and configurable human editing interface. Project-team usability testing this past winter and earlier this summer, and professional usability testing to commence late this month OneStop user interface V1.1 and supporting API now operational and in “soft launch” period (note, this URL will change). V1.2 release imminent.

“OneStop Ready” Data Group Status Percent Ready Collection Metadata Granule Metadata Data Formats Data Access DSMM PMM (optional) Digital Elevation Models 96% 90% 100% CO-OPS NWLON PORTS 98% World Ocean Atlas 2013 Group for High Resolution SST 95% NDBC C-MAN NOAA Climate Data Records 97% Y OCS Hydro COAPS SAMOS 94% NEXRAD Level 2 and 3 86% 30% Reformatted Legacy GOES GVAR data S-NPP/JPSS 91% 75% 85% Water Column Sonar Data 41% 74% 50% 0% 80% ESSA images 35% 15% 10% Hazard Images Database 54% 40% Global Historical Climatology Network - Daily, Version 3 48% GOES-16 Space Weather (excluding SUVI) 34% GOES-16 ABI L1b Radiances and L2 CMIP 62% 20% World Ocean Database 79% 65% 70% 60% DSMM - Data Stewardship Maturity Model PMM - Product Maturity Model Version 1.0 readiness status, as of 01 Sep 2017

Key Notes for WGISS OneStop will soon replace the existing Geoportal-based connection to CWIC for the Group for High Resolution SST (GHRSST) data - perhaps in the December release In addition, new data will become available: by December, about 15 groups will be available following OneStop best practices, which include the generation of both collection level and granule-level metadata. Slide 9 lists the groups being worked on now… they include many S-NPP products, reprocessed historical GOES, NEXRAD, some GOES-16 products, several in situ data sets, and even rescued and reformatted ESSA satellite images from the 1960s/1970s A longer term goal is to have granule metadata for all archived data, including all of the current holdings of CLASS OneStop currently also includes collection level records for all archived data, and will eventually include collection records for all NOAA data

OneStop User Interface