Presentation is loading. Please wait.

Presentation is loading. Please wait.

December 10, 2010. National Climatic Data Center Background System Development Operational Integration Recent Accomplishments Future Direction Next Steps.

Published byReynold Atkinson Modified over 8 years ago

Similar presentations

Presentation on theme: "December 10, 2010. National Climatic Data Center Background System Development Operational Integration Recent Accomplishments Future Direction Next Steps."— Presentation transcript:

1 December 10, 2010

2 National Climatic Data Center Background System Development Operational Integration Recent Accomplishments Future Direction Next Steps December 10, 2010 DAARWG Meeting: CLASS Update 2

3 National Climatic Data Center No place to put mountain of data. Manage rapidly growing data volume of major observing and modeling systems Every program solving the same problem. Eliminate various "stove-pipe” systems and produce a unified "enterprise” data access system to reduce IT cost Satellite Active Archive (SAA) GOES Active Archive (GAA) Earth Observing System (EOS) Archive Can’t access the data we have. Centralize NOAA’s numerous data systems for environmental data access-- create a single portal Don’t break anything. Retain, as much as possible, portions and modules of existing legacy systems December 10, 2010 DAARWG Meeting: CLASS Update 3

4 National Climatic Data Center Scope. Enterprise-wide IT system supporting long- term, secure storage of and common access to environmental datasets and information stewarded by NOAA’s Archives. 1.Large Data Campaigns. Satellites (NPP/JPSS, GOES, POES, DMSP, MetOp), Radar (NEXRAD), Models 2.Enterprise Approach Providing common services for development and operation of IT systems supporting NOAA Archives Consolidating legacy archival storage systems Relieving data producers of responsibility for archival December 10, 2010 DAARWG Meeting: CLASS Update 4

5 National Climatic Data Center The CLASS support contract was awarded competitively on June 20, 2008 to Diversified Global Partners (DGP) JV LLC Small business set-aside – 8(a) mentor- protégé program Protégé Company: DB Consulting Group Mentor Company: Global Science and Technology (GST) Inc. Potential nine-year period of performance – Now in year 3: Base Year Four (4) one-year Option Periods Four (4) one-year Award Term Option Periods Indefinite Delivery/Indefinite Quantity (IDIQ) contract Maximum Ordering Volume of $200M (maximum of nine years) Cumulative Tasking to date valued at $42.0M December 10, 2010 DAARWG Meeting: CLASS Update 5 NGDC (Ops) Boulder, CO NCDC (Ops) Asheville, NC Fairmont, WV (Devel) NSOF (Devel/Ingest) Suitland, MD

6 National Climatic Data Center Systems Software Data Integration December 10, 2010 DAARWG Meeting: CLASS Update 6

7 National Climatic Data Center CLASS Software Evolution (CLASS-SE). Five-year project to establish configurable ingest NOAA Enterprise Archive Access Tool (NEAAT). Enterprise Application Program Interface (API) December 10, 2010 DAARWG Meeting: CLASS Update 7 Stakeholders CLASS System Producer Consumer Ingest Preservation Planning Administration Data Management Archival Storage Access System Administrators Data Stewards OAIS-RM

8 National Climatic Data Center CLASS-SE provides configurable ingest capability Configurability reduces development costs for storage and access of new data types Applications, or services, are ‘data agnostic’ and may be applied against selected data types Workflow engine supports both Collective system operations & resource allocation Allocates services from 1 to N instances of CLASS nodes Individual Nodes may perform all or sub-set of system capabilities December 10, 2010 DAARWG Meeting: CLASS Update 8

9 National Climatic Data Center Enterprise access to all NOAA archive storage systems Satisfies L1RD Requirements CLASS Access Interface Support to GEO/IDE Interface to Legacy Systems Supports both data access and stewardship applications Service-Oriented Architecture (SOA) Middleware Simple plugin adaptor in integration layer provides interface to NEAAT Support for open source tool kits (i.e., OGC) OPeNDAP protoype provides access to climate model data data through NOMADS (National Operational Model Archive & Distribution System) December 10, 2010 DAARWG Meeting: CLASS Update 9 Legacy Systems NEAAT Plugin Satellites CFSR CLASS NARR NOMADS NCDC HDSS NCEP Models ESG INE IDEAS ESSE ESG INE IDEAS ESSE SPIDR Plugin Standard Protocols (OPeNDAP) Standard Protocols (OPeNDAP) Customer Application Customer Application Data Migration

10 National Climatic Data CenterDecember 10, 2010 DAARWG Meeting: CLASS Update 10 Data SetPhaseStatus GOEScompleteoperational POEScompleteoperational DMSPcompleteoperational MetOpcompleteoperational EOS MODIS-canceled Jason 1&2completeoperational Jason 3not startedundefined, expected volume & complexity NPPin progresssee next slide JPSS (NPOESS)planningrequirement added to JPSS L1RD GOES-Rin progress Completed Systems Definition Review & Systems Requirement Review, preparing for Preliminary Design Review NEXRADplanning NCEP ModelsplanningCFSR complete, need to refine requirement

11 National Climatic Data CenterDecember 10, 2010 DAARWG Meeting: CLASS Update 11

12 National Climatic Data Center First climate model data archived on CLASS; access through NOMADS 245 TB of CFS reanalysis in tape archive 100 TB of “high-priority” data available on disk for rapid access Just began ingest of reforecast NCDC-NCEP-CLASS Project partnership Reuse of existing spinning disk Agile Software Development; rapid integration of open source access protocol One of NCEPs most important data sets Significant jump in NCDC data access December 10, 2010 DAARWG Meeting: CLASS Update 12

13 National Climatic Data Center First NOS data set in CLASS Complete: CORS File Naming Convention document signed May 2010 NGDC has established a centralized ingest interface to CLASS In-process: Interface Control Document CORS goals for CLASS version 5.4 release: archive of forward-looking RINEX files (3173 daily files) and metadata daily ingest ~ 49.5 GB/day Future CORS goals, pending success of CLASS archive: archive of forward-looking binary files archive of historical RINEX and binary files archive of NGS reanalysis data current NGDC archive total: ~69.0 TB December 10, 2010 DAARWG Meeting: CLASS Update 13

14 National Climatic Data CenterDecember 10, 2010 DAARWG Meeting: CLASS Update 14 Simple model based on preservation through two-site replication. Ingest Node Ingest Node NSOF Full Node Full Node NCDC Full Node Full Node NGDC Replication Archive Data Sources

15 National Climatic Data CenterDecember 10, 2010 DAARWG Meeting: CLASS Update 15 Data Stewardship Data Stewardship Sub-Node (Federated) Sub-Node (Federated) Center of Data Center of Data Processing Node Processing Node Data Processing Data Processing Ingest Node Ingest Node Data Producer Data Producer Full Node Full Node Data Center Data Center Cloud Access Increase distribution of nodes Federate with Centers of Data providing tiers of service Exploit cloud resources for faster access Becoming more H/W agnostic

16 National Climatic Data Center NCDC partnership with RENCI and DataNet Consortium Prototype “system of systems” framework; federation of NOAA data systems with NOAA archive using iRODS (Integrated Rule-Oriented Data System) Connectivity to data systems such as RENCI, ORNL, OOI, and Earth System Grid (ESG) Pilot Project Federate with RENCI to share 70TB of NEXRAD data Utilize highly distributed computing to derive climate- quality precipitation re-analysis, push data products to NCDC archive system (CLASS) Future Plans to Support Climate Assessments Federate with NOS systems via RENCI to integrate data from OOI with climate data at NCDC Federate with GFDL and ESG to integrate climate model data with in situ and satellite data at NCDC December 10, 2010 DAARWG Meeting: CLASS Update 16

17 National Climatic Data Center #1 Priority: NPP Operational Test & Evaluation Prepare FY13 Submission Release of 5.4.1; implement final version of NEAAT Complete Cloud Computing Study Establish Archive Architecture and ConOps Prepare for Transition to Climate Service Stand-up Project Management Staff Program Review Migrate data from legacy systems December 10, 2010 DAARWG Meeting: CLASS Update 17

18 National Climatic Data Center Need for NOAA-level focus on enterprise infrastructure (beyond “comm-lines”), i.e., NOAA Program for GEO-IDE development and fielding Need for NOAA-level policies, directives, and concepts to constraint operational practices and guide IT investments, i.e., CLASS December 10, 2010 DAARWG Meeting: CLASS Update 18

19 Scott Hausman Acting Director NOAA’s National Climatic Data Center (NCDC) 151 Patton Avenue, Room 557 Asheville, NC 28807-5002  828-271-4848  828-271-4246  828-450-9188 Scott.Hausman@noaa.gov www.ncdc.noaa.gov

Download ppt "December 10, 2010. National Climatic Data Center Background System Development Operational Integration Recent Accomplishments Future Direction Next Steps."

Similar presentations

NOAAs Comprehensive Large-data Array Stewardship System (CLASS) Robert Rank NOAA CLASS Program Chris Elvidge – NOAA-NGDC January 22, 2006.

NOAAs Comprehensive Large-data Array Stewardship System (CLASS) Robert Rank NOAA CLASS Program Chris Elvidge – NOAA-NGDC January 22, 2006.
NOAA Report David Clark NOAA/NESDIS/NGDC CEOS/WGISS 20 Budapest, Hungary May 12, 2006.

NOAA Report David Clark NOAA/NESDIS/NGDC CEOS/WGISS 20 Budapest, Hungary May 12, 2006.
Comprehensive Large Array-data Stewardship System Status Presented to DAARWG Kern Witcher CLASS Program Manager November 8, 2012.

Comprehensive Large Array-data Stewardship System Status Presented to DAARWG Kern Witcher CLASS Program Manager November 8, 2012.
© 2009 The MITRE Corporation. All rights Reserved. Evolutionary Strategies for the Development of a SOA-Enabled USMC Enterprise Mohamed Hussein, Ph.D.

© 2009 The MITRE Corporation. All rights Reserved. Evolutionary Strategies for the Development of a SOA-Enabled USMC Enterprise Mohamed Hussein, Ph.D.
1 2008 AMS Summer Community Meeting Boulder, Colorado – 11-13 August 2008 The Emerging Interface among the Private, Federal & Academic Sectors in the Archive.

AMS Summer Community Meeting Boulder, Colorado – August 2008 The Emerging Interface among the Private, Federal & Academic Sectors in the Archive.
Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.

Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.
1 Comprehensive Large Array-data Stewardship System (CLASS) GOES-R -- Overview and Plans Richard G. Reynolds CLASS Project Manager.

1 Comprehensive Large Array-data Stewardship System (CLASS) GOES-R -- Overview and Plans Richard G. Reynolds CLASS Project Manager.
Federal Aviation Administration Presented to: Network Centric Demonstrations and Prototypes Technical Interchange By: Deborah Young Date: November 4, 2009.

Federal Aviation Administration Presented to: Network Centric Demonstrations and Prototypes Technical Interchange By: Deborah Young Date: November 4, 2009.
May 17, 2006 1 Capabilities Description of a Rapid Prototyping Capability for Earth-Sun System Sciences RPC Project Team Mississippi State University.

May 17, Capabilities Description of a Rapid Prototyping Capability for Earth-Sun System Sciences RPC Project Team Mississippi State University.
Getting to know IRMA ( Integration of Resource Management Applications) - Overview By: Margaret Beer, Brent Frakes, Alison Loar, Simon Kingston National.

Getting to know IRMA ( Integration of Resource Management Applications) - Overview By: Margaret Beer, Brent Frakes, Alison Loar, Simon Kingston National.
NOAA National Climate Model Portal D. Michael Grogan Lead Systems Engineer National Climatic Data Center.

NOAA National Climate Model Portal D. Michael Grogan Lead Systems Engineer National Climatic Data Center.
NOAA Metadata Update Ted Habermann. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International.

NOAA Metadata Update Ted Habermann. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.

System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
Data-PASS Shared Catalog Micah Altman & Jonathan Crabtree 1 Micah Altman Harvard University Archival Director, Henry A. Murray Research Archive Associate.

Data-PASS Shared Catalog Micah Altman & Jonathan Crabtree 1 Micah Altman Harvard University Archival Director, Henry A. Murray Research Archive Associate.
A.V. Bogdanov Private cloud vs personal supercomputer.

A.V. Bogdanov Private cloud vs personal supercomputer.
DYNAMICS CRM AS AN xRM DEVELOPMENT PLATFORM Jim Novak Solution Architect Celedon Partners, LLC

DYNAMICS CRM AS AN xRM DEVELOPMENT PLATFORM Jim Novak Solution Architect Celedon Partners, LLC
Scott Hausman Acting Director, NOAA National Climatic Data Center (NCDC), Asheville, North Carolina June 3, 2010.

Scott Hausman Acting Director, NOAA National Climatic Data Center (NCDC), Asheville, North Carolina June 3, 2010.
Field Project Planning, Operations and Data Services Jim Moore, EOL Field Project Services (FPS) Mike Daniels, EOL Computing, Data and Software (CDS) Facility.

Field Project Planning, Operations and Data Services Jim Moore, EOL Field Project Services (FPS) Mike Daniels, EOL Computing, Data and Software (CDS) Facility.
Data Interoperability and Access Activities Prepared for the Data Archiving and Access Requirements Working Group (DAARWG) Ken McDonald, TPIO/GEO-IDE Jeff.

Data Interoperability and Access Activities Prepared for the Data Archiving and Access Requirements Working Group (DAARWG) Ken McDonald, TPIO/GEO-IDE Jeff.
NE II NOAA Environmental Software Infrastructure and Interoperability Program Cecelia DeLuca Sylvia Murphy V. Balaji GO-ESSP August 13, 2009 Germany NE.

NE II NOAA Environmental Software Infrastructure and Interoperability Program Cecelia DeLuca Sylvia Murphy V. Balaji GO-ESSP August 13, 2009 Germany NE.

Similar presentations

Ads by Google