1 Comprehensive Large Array-data Stewardship System (CLASS) GOES-R -- Overview and Plans Richard G. Reynolds CLASS Project Manager May 12, 2004

2 Agenda CLASS System Overview Project Management Accomplishments to Date Current Hardware Architecture FY04 Goals “Final” Architecture Budget … Resources and Commitments Issues and Risks

3 CLASS Overview NOAA's National Data Centers and their world-wide clientele of customers look to CLASS as the sole NOAA IT infrastructure project in which all current and future large array environmental data sets will reside. CLASS provides permanent, secure storage and safe, efficient access between the Data Centers and the customers.

4 CLASS Overview CLASS is a web-based data archive and distribution system for NOAA/NESDIS environmental data Archive … ingest, storage, metadate management, and data quality assurance Distribution … access, visualization, and data delivery CLASS is an extension of an 1995 operational system … SAA (Satellite Active Archive) –Transition to the CLASS architecture began in 2001 –“Dual-Site” Operational CLASS began 02 April 2004 CLASS currently supports POES, DMSP, and GOES data sets, plus RadarSat (Synthetic Aperture Radar) and SeaWiFS (Ocean Color Product) CLASS will support additional campaigns, broader user base, new functionality currently being defined –CLASS concurrently support ongoing operations and new requirements implementation

5 System Overview – Functional Flow Diagram Ingest and Store Data Visualization Data Data Set Inventory Data Caches Orders Maintain, Monitor, Control Process Orders Access Data Visualize Data Interface with Users Data Products And Metadata Data Providers USERS CLASS Operators Archive CLASS Internet/Intranet

6 System Overview – System Design Data Storage and Distribution Subsystems – Ingest, Subscription and Bulk Order Generation, Data Recall, Delivery Servers: –Inventory, Visualization, Order Socket messages in XML format Object oriented design implemented in C++

7 System Overview – System Design User Interface –Java/XML-based web interface uses COTS components Apache – Web Server; Cocoon - publishing framework Avalon/Excalibur - database connection pooling Tomcat - servlet engine LogKit - message logging Informix Java Database Connectivity (JDBC) Driver Monitoring and Maintenance Tools –Log Monitor, Cache Cleanup, Work Space Cleanup, Independent Monitoring, Operator Interface C++ Perl

8 System Overview – Distributed Redundant Sites Ingest process Operational inventory Archiver Archive interchange Robotic storage Supplier Ingest process Operational inventory Archiver Archive interchange Robotic storage Suitland Asheville Operational datastore USER

9 System Overview – Servers

10 System Overview – User Interface USERS

11 System Overview – Maintenance and Monitoring

12 CLASS Overview (Continued) NOAA Observing System -- Notional Architecture -- The “End-to-End System” Larger System Observing System Data Handling System Human Environmental Phenomenon Environmental Parameter Sensing Element Sensor Platform / Station part of measures is a contains characterizes provides data to Observation Control System is controlled by Location located at MobileFixed is Space Air Ground Ocean Space Air Ground Ocean Basic Service Requirement < drives provides data directly to User provides info to is type of Stake- holder has Operator operated by situated on Support supported by Owner owned by Stakeholder Requirement < drives Processing Element RemoteIn Situ is type CLASS

13 Project Management NESDIS Data Archive Board CLASS Project Richard G. Reynolds CLASS Project Management Team (CPMT) NGDC Development Teams (Boulder, CO) NCDC Development Team (Fairmont, WV) OSDPD Development Team (Suitland, MD) System Integration & Test Team (Suitland, MD) OSDPD Operations (Suitland, MD) NCDC Operations (Asheville, NC) Archive Requirements Council ITAT

14 Richard G. Reynolds (OSD) Charles Bryant (OSD), Chung Wu (OSD) Suitland Team Alex Kidd (OSD) Robert Rank (OSD) Tino Cremidis (CSC) David Bowman (NCDC) Baseline system Processes/policies System Architecture & Design System Integration & Testing Support Suitland operations West Virginia Team Carlos Martinez (TMC) GOES Lead CCR Implementation NPP Implementation EOS Implementation Support Asheville operations Boulder Team Eric Kihn (NGDC) SABR Integration DMSP Data Mining Ted Habermann (NGDC) Geospatial databases Geotiff images Metadata COAST Chung Wu (OSD) E-commerce Order Management System (Continued) Project Management (Continued)

15 Data Archive Board (DAB) (Tom Karl/NCDC) CLASS PROJECT OSD (w/ CSC & TMC) & Data Centers (w/ Contractors) Archive Requirements Council (ARC) (John Bates/NCDC) Information Exchange

16 CLASS Accomplishments to Date Completed overall design of CLASS top-level architecture Prepared Key System Documentation –Requirements –ICDs –CONOPS –Management Plans and Procedures Established operational, integration and test, and development environments in Suitland Completed migration from SAA to CLASS –Migrated about 30TB of data from old tape archive system –Added more than 50 new derived products to the archive Consolidated three web sites in one web-based user interface

17 CLASS Accomplishments to Date (Continued) Enhanced ingest system to be independent of file type Delivered baseline systems to Suitland and Asheville Suitland CLASS Operational with POES and DMSP data sets Completed IJPS/Metop – Archive and Access Segment Preliminary Design Review Coordinated with NPP/NPOESS for defining the IDPS to CLASS Interface Control Document (draft) Worked with NASA personnel to define initial requirements to archive EOS/MODIS Level-0 data.

18 CLASS Accomplishments to Date (Continued) Established the CLASS Operations Team (COT) –Includes OSDPD and NCDC personnel –In preparation for Baseline Dual-Site Operations Established a project wide risk management program Completed documentation of business case for CLASS Suitland CSC Development Team Certified –SEI-CMM Level-3 Worked with IBM-China for possible implementation of CLASS for National Satellite Meteorological Center (NSMC)

19 CLASS Accomplishments to Date (Continued) Completed … Summary “10-year” CLASS Project Budget –FY04/05, plus $25.6M Flat-line …. FY06-FY14 Completed … NPP/NPOESS Campaign Plan Completed … FY04 Spendplan –Against the Approved $10.5M NESDIS funding allocation Completed … Initial CMMI Training at TMC Completed … CLASS Developers Workshop –Held 03/04 March 2004 in Suitland –Brought together all CLASS team members

20 CLASS Accomplishments to Date (Continued) Completed … Software Release 2.0 / 2.1 / 2.2 –Operational Dual-site configuration –Operational 02 April 2004 Completing GAA Activities –Final McIdas-less software design in final preparation at TMC Planning for EOS/NPP Implementation at TMC –Task SOW Completed, sent to TMC, and Proposal received –NASA/IV&V Facility MOU in preparation for floor space

21 CLASS -- Near-term Plans CLASS Release 3.0 –Scheduled for 24 May 2004 –Provides: Delivery Manifest and Web Enabled Subscription Management CMMI Activities –Preparing all CLASS development teams for Internal Evaluation in July 2004 CLASS Release 3.1 –Scheduled for August 2004 –Provides: Ingest Enhancements to support IJPS NOAA data

22 CLASS -- Near-term Plans (Continued) System Storage Capacity Upgrade –Scheduled for September 2004 –Additional disk space at both CLASS operational sites CLASS Release 3.2 –Scheduled for November 2004 –Provides: Ingest enhancements to support IJPS EUMETSAT data, and IJPS Test readiness

23 FY04 CLASS Goals Prepare a “5-Year” Budget and Program Plan for CLASS (COMPLETED) Focus CLASS resources on CLASS Activities (COMPLETED) –CLASS, GAA, EOS, SAA Projects combined Initial test of CLASS enhancements for IJPS … NOAA-N (October COMPLETED) Configure and Test CLASS at NCDC and transition to Stand-alone Operations (December COMPLETED) Complete testing and transition to operations of CLASS dual site capabilities (April COMPLETED) Complete testing and transition to operations of the GOES Campaign (April COMPLETED) Complete IJPS/Metop Critical Design Review (21 April COMPLETED)

24 FY04 CLASS Goals (Continued) Configure and Test CLASS for support of IJPS/Metop, and transition to Operations (September 2004) Establishment of an integrated Design-Implementation team between Suitland and TMC/Fairmont Design NPP Campaign Data Ingest as initial phase of NPP Campaign, and with partial implementation Design and Implement EOS Data Ingest as initial phase of EOS Campaign Design of the JASON Campaign Data Ingest, as initial phase of the JASON and OSTM Campaign(s) Completion of the GOES (development) Campaign Continue migration of GOES Retrospective Data Sets into CLASS Continue migration of POES Retrospective Data Sets into CLASS O&M of Dual-site CLASS System at Suitland and Asheville

25 Current CLASS Hardware Architecture CLASS hardware for Suitland and Asheville sites: –Operational Servers, 4 at each site IBM P660, 4GB RAM 2CPUs –Integration and Test Servers, 2 at each site IBM P660 –Development Servers, 2 at Suitland IBM P660 –Storage Area Network (SAN) SUN 9980, 5.2TB –Tape Archive System & Hierarchical Storage Manager (HSM) 9522 LTO Tape Slots, 12 Drives, 8PB LTO-3

26 Suitland System Hardware

27

28 The Future CLASS

29 CLASS Budgets FY00 and prior N/A FY01$1.995M* FY02$3.599M* FY03$2.881M* FY04$10.5M FY05 $11.1M * NET FY06$11.9M FY07$9.9M FY08$9.9M FY09$22.1M FY10+ $27.9M/yr

30 “GOES-R Campaign” Requirements Definition and ICDs Data and Products Ingest Storage, Processing, and Communications Upgrades Metadata extension Catalogue extension Visualization extension Reprocessing extension

31 CLASS “GOES-R” Budget FY07$2.0M FY08$3.8M FY09$4.1M FY10$7.6M FY11$8.5M FY12$6.9M FY13$6.2M FY14$2.0M $41.0M TOTAL

32 THANK YOU!