Presentation is loading. Please wait.

Presentation is loading. Please wait.

Comprehensive Large Array-data Stewardship System Status Presented to DAARWG Kern Witcher CLASS Program Manager November 8, 2012.

Similar presentations


Presentation on theme: "Comprehensive Large Array-data Stewardship System Status Presented to DAARWG Kern Witcher CLASS Program Manager November 8, 2012."— Presentation transcript:

1 Comprehensive Large Array-data Stewardship System Status Presented to DAARWG Kern Witcher CLASS Program Manager November 8, 2012

2 Agenda CLASS Summary Program Overview Data Center Migration CLASS Capabilities CLASS System Evolution Other Initiatives Challenges CLASS and DAARWG 2

3 CLASS Summary CLASS meets original intended purpose (Large Volume Data) Suomi-NPP Volume Ingested – 1.32Pb Volume delivered-3.53Pb Files Delivered- over 34M 22 major datasets included AVHRR, GOES, IASI, CFS-R 2.27 Pb, safely stored at 2 sites CLASS must evolve to become a sustainable NOAA enterprise NOAA’s desire for CLASS to be the Enterprise Archive Significant challenges must be addressed 3

4 Program Overview 4

5 Program Transition Program/ Contract Management was transitioned from OSD to NCDC in May of 2011 New WBS and program management staff established Increased communications between CLASS development and Data Centers CY3* extension in Fall of 2011 to end of FY12 CY4 extension though Q1 of FY13 CY5 to begin Q2 of FY13. CY5 delayed to allow for CLASS response to emerging Data Center consolidation data migration activities begin “pivot” or evolution of CLASS to the Enterprise Archive System Effort is dependent on final funding allocations for FY13 Contract may reach ceiling as early as FY15 Acquisition planning needs to begin by January *- Contract Year

6 Organization NGDCNODC CLASS Operations Planning Board COWG NCDC Other NESDIS Contractor Legend Supervisory Reporting Remote Sensing Applications Division CLASS Program Manager – 100% (Kern Witcher) NCDC IT Specialist/Systems Engineer - 100% (Jay Morris ) Support Services Division Budget Analyst– 5% (L. Cholid) Inventory Spec – NCDC – 10% (A. Annis, acting ) OMB Exhibit 300 – 30% (T. Cohen) Budget Execution - 5% (T. Leary) COR / Proj Engineer + CWIP – 100% (Jim Goudouros ) ISSO – 100% (Scott Koger) Alt COR – 5% (J. Niemiec ) Government Task Monitor – 30% (N. Ritchey) Operations Manager – 100% (D. Carter)) NESDIS CIO Total of 5 dedicated personnel 6

7 Current Program Milestones 7 Currently focused on NESDIS and NWS data sets

8 Ongoing Challenges Architectural Boundary Responsibilities Common vs. Unique Data Ingest Data Center Migration Data Center Consolidation Governance Budget Transition to New Architecture 8

9 CLASS Definition Purpose: Support long-term, secure storage of NOAA-approved data, information, and metadata and enable access to these holdings through both human and machine-to-machine interfaces. Capabilities will be provided in 3 primary functional areas (Open Archive Information System Reference Model, OAIS-RM components): –Ingest - provide mechanisms by which data, information, and metadata are transferred to and organized within the storage system. Issue: Feasibility – Cannot afford/maintain unique solutions 9 From CLASS Level 1 Requirements (preliminary) signed 2008

10 CLASS Definition (con’t) Archival Storage - provide common enterprise means for data, information, and metadata to be stored by the system and the capability to refresh, migrate, transform, update, and otherwise manage these holdings as part of the preservation process. Access - provide common enterprise access capability enabling users to identify, find, and retrieve the data and information of particular interest to the user. Issue: Open to interpretation – IT systems versus Data Center stewardship responsibilities Optimal data access varies by observations (vertical profiles vs. spatial patterns, Gridded vs. swaths, Time series vs. synoptic) 10

11 11 Goals: As an enterprise solution, CLASS will reduce anticipated cost growth associated with storing environmental datasets by: –Providing common services for acquisition, security, and project management for the IT system supporting NOAA Archives –Consolidating stove-pipe, legacy archival storage* systems –Relieving data owners of archival storage-related system development and operations issues Archival storage provides the services and functions for the storage, maintenance and retrieval of archival information packets. Archival storage functions include receiving archival information packets from ingest and adding them to permanent storage, managing the storage hierarchy, refreshing the media in which archive holdings are stored, performing routine and special error checking, providing disaster recovery capabilities, and providing archival information packets to access to fulfill orders 11 CLASS Goals

12 High Level Requirements for CLASS 5.1 Core Mission Requirements CLASS shall provide defined and documented human and machine-to- machine interlaces by which archives may securely store, maintain, and provide access to their data, information, and metadata holdings for indefinite periods CLASS shall ingest, provide long-term, secure storage, and provide access to baseline information holdings CLASS shall provide long-term, secure storage of and common access to information pertaining to processing of CLASS maintained information holdings, including documentation, processing algorithms, and procedures CLASS shall comply with applicable National Archives and Records Administration (NARA) regulations CLASS shall initiate pilot programs with the GEO-IDE project to support risk reducing development and phased integration of standards for metadata, machine- to-machine interlaces, and archives. From CLASS Level 1 Requirements (Preliminary 2008) 12 Proposed NOAA Action: Finalize CLASS Level 1 Requirements

13 Current CLASS Governance CLASS is designated a "major" NOAA system in accordance with criteria in NOAA's Administrative Order (NAO) NOAA Observing Systems Council (NOSC) is the designated oversight council and will review the CLASS project at each Key Decision Point (KDP). The project will be reviewed as a major project by the Program Management Council (PMC), unless delegated. As an IT project, the CLASS project will be reviewed by the NOAA Information Technology Review Board (NITRB) and by the Commerce Information Technology Review Board (CITRB). Changes to project baseline, scope, and direction shall be approved by the Deputy Undersecretary for Oceans and Atmosphere From the CLASS Level I Requirements 13 The CLASS Program has numerous oversight and approval authorities Proposed NOAA Action: Provide Clear Lines of Authority and a well defined Governance Structure

14 Data Center Migration 14

15 Goal, Objective, and Outcomes from NOAA Data Centers Data Migration Plan v1.2 signed May 2011 Goal: Prior to the end of FY 2015, the CLASS Operational System will be the primary safe storage/access capability for all environmental data holdings under the auspices of NOAA’s Data Centers. Objectives: The objective of this plan is migration of all historical environmental data holdings on current Data Center storage systems to CLASS, in addition to the ingest into CLASS [of] near real-time environmental data streams currently received by the Data Centers. This objective involves archival storage as well as elements of ingest, data access, data management, and preservation planning. Data Center Migration Status 15

16 Starting point Incoming Data Streams already migrated to CLASS include – POES, GOES, Jason-2, MetOp Incoming Data Streams solely in CLASS – Suomi-NPP, GCOM-W, (future GOES-R, future JPSS, future Jason-3) Data Center Migration Status(con’t) 16

17 Data Center Status Each Data Center developing its own individual plans including schedule, metrics and milestones CLASS PM included several pilot activities in CLASS’ Contract Year 4 (CY4) – Cloud access pilot (NODC, NCDC) - ongoing – NEXRAD pilot (NCDC) - completed – NCDC in situ migration prototype activity - completed Next steps to be included in CLASS’ CY5 activities Data Center Migration Status(con’t) 17

18 Data Center Migration Status(con’t) CLASS accomplishments in FY12 for Data Center Migration Aerospace completed Phase I (“as is” analysis) of Data Center Con Ops Completed Data Center Migration Requirements Documents Completed Data Center Migration Interface Control Documents Migrated NCDC Historical Data Set Completed NexRad Pilot 2 week migration test Initiated Cloud Pilot Project 18

19 Uncertainty regarding FY15 timeline – Reductions in CLASS funding available for migration activity could delay implementation Increasing comms costs, operational costs FY13 President’s Budget has increased ORF to solve this – Data Center consolidation activities – Individual Data Center plans not yet finalized Several possible technological solutions still being evaluated with Data Centers and CLASS Data Center Migration Status(con’t) 19

20 Summary The Data Centers have an overall plan in place for the migration of their historical environmental data to CLASS by FY15. Many historical data sets already migrated – 60% by volume – but many smaller sets still need to be completed. Several large data streams already migrated, but many incoming data streams still need to be transitioned to CLASS storage. Each Data Center working on their individual plans and coordinating with CLASS PM. CLASS PM will address next steps in FY13/CY5, contingent upon available funds for migration activities. Data Center Migration Status(con’t) 20

21 Current Capabilities 21

22 Current CLASS Assets CLASS NGDC, Boulder CO CLASS NSOF, Suitland MD CLASS NCDC, Asheville NC Replication via NOAA Science Network(N-WAVE) Users Development Team Fairmont, WV Functions Operations Ingest Storage (Disk & Tape) Public Access Test and Integration Environment Satellite Landing Zone Development Environment Direct Connectivity to: ESPC- NOAA Environmental Satellite Processing Center National Ice Center NOAA Coast Watch JPSS Interface Data Processing Segment (IDPS) Key Capabilities: Tape Library Capacity – 2- 10,000 tape robotic libraries with a total storage capacity of 15.5 Pb (LT04 Tapes/ Native) Spinning Disk Capacity- 2.1 Pb (NSOF, NCDC, NGDC) 10 Gb/sec Internal Network Backbone Redhat Linux OS Server Count- 48 (576 processors) 10 Gb/Sec WAN (N-Wave) Key Capabilities: Tape Library Capacity – 2- 10,000 tape robotic libraries with a total storage capacity of 15.5 Pb (LT04 Tapes/ Native) Spinning Disk Capacity- 2.1 Pb (NSOF, NCDC, NGDC) 10 Gb/sec Internal Network Backbone Redhat Linux OS Server Count- 48 (576 processors) 10 Gb/Sec WAN (N-Wave) 22

23 CLASS interfaces to NPP and JPSS-1 OPS RN I&T RN OPS RN I&T RN NWAVE OPS FSN Boulder OPS FSN Asheville = Receipt Node – Data delivery to archive = Full Service Node (Ingest), Archive, and User Access Primary JPSS IDPS Backup IDPS at CBU NSOFFairmont = CLASS backbone WAN GRAVITE IFCs Backup system for COOP Event FUTURE IDPS Block 1.5/2.0 NPP, J-1, J-2 CLASS NGDC CLASS NCDC STAR CLASS NSOF SDS IDPS GRAVITE NWAVE 10Gb/s C3S CURRENT IDPS Block 1.2 NPP 23

24 CLASS Interface to GOES-R GOES-R (In Development) 24

25 Data Access from CLASS 25 Preservation Planning Administration Ingest Access Data Management ArchivalStorage PRODUCERPRODUCERPRODUCERPRODUCER CONSUMERCONSUMERCONSUMERCONSUMER requests results Present CLASS

26 NPP Data Dissemination CLASS NGDC CLASS NCDC STAR CLASS NSOF SDSIDPS GRAVITE CLASS to SDS Near-instantaneous subscription delivery (~3.3 TB/day) NWAVE 10Gb/s Public Subscriptions Public Ad hoc orders IDPS to CLASS 6-hour delay C3S Svalbard, Norway ~4.7TB/day CLASS to STAR Near-instantaneous subscription delivery (~3.3 TB/day) CLASS to GRAVITE Near-instantaneous subscription delivery (~3.3 TB/day) Replication ~20-min 6-hour Delay imposed by CLASS 1)Minimizes retransmissions requests 2)Ensures control of limited distribution files. Public Subscriptions Near-instantaneous delivery Public Ad Hoc Block Orders (up to 18 Tb/day) 50% complete within 0 to 6 hours 27% complete within 6 to 12hours 22% complete within 12 to 72 hours 1% complete over 72 hours Public Ad Hoc Bulk Orders Up to 7 days ProcessDuration IDPS to CLASS~ 6 hours File Ingest & catalog populated~ 20 min NCDC/NGDC Node Replication~ 22 min Total~ 6:42 Note: Near- Instantaneous subscription delivery is limited by network bandwidth 26

27 Data Currently Archived in CLASS 27 Dataset NamePeriod of Record Date First Archived into CLASS Total Volume (TB)Source Suomi National Polar-orbiting Partnership (S-NPP)10/2011 to present10/1/ NASA/NOAA Geostationary Operational Environmental Satellite (GOES)1/1979 to present12/5/ NOAA Climate Forecast System - Reanalysis (CFS-R)1/1979 to 3/20111/20/ NOAA Advanced Very High Resolution Radiometer (AVHRR)10/1978 to present11/1/ NOAA Infrared Atmospheric Sounding Interferometer (IASI)4/2007 to present5/1/ EUMETSAT Defense Meteorological Satellite Program (DMSP)7/1987 to 2/199711/1/ DoD Advanced Clear-Sky Products over Oceans (ACSPO)5/2008 to present5/1/ NOAA Coast Watch2/1989 to present11/15/ NOAA Advanced Scatterometer Level 1B (ASCAT)2/2007 to present2/1/ NASA Synthetic Aperture Radar (SAR)6/1992 to 3/20114/30/ NASA/NOAA Continuously Operating Reference Stations (CORS)11/2010 to present10/7/ NOAA Jason-26/2008 to present6/22/ NASA Microwave Integrated Retrieval System (MIRS)8/2007 to present8/30/ NOAA Ocean Color Data Products5/2000 to present5/7/ NOAA Tiros Operational Vertical Sounder (TOVS)1/1988 to 12/199511/1/ NOAA Global Nav Satellite Receiver for Atmospheric Sounding Level 1B (GRAS)9/2007 to present9/14/ EUMETSAT Microwave Surface and Precipitation Products System (MSPPS)7/2003 to present7/20/ NOAA Aerosol Optical Thickness (100 KM) (AERO100)11/1998 to present11/15/ NOAA Solar Backscatter Ultraviolet Spectral Radiometer Version 23/1985 to present5/23/ NOAA Global Change Observation Mission 1 - Water (GCOM-W1)8/2012 to present8/22/ JAXA Coral Bleaching Monitoring Products (CORBL)6/2007 to present6/6/ NOAA NCEP Weather Analyses and Forecast Charts3/1999 to 5/20125/2/ NOAA Total Represent 95% of the Archival Holdings by Volume (Single Node)

28 Customer Groupings that Access CLASS CLASS Users by Domain TB

29 Current Constraints 29 Access limited through CLASS Web site New data sets to archive require significant SW development efforts Implementation of Trusted Internet Connection (TIC) will effect throughput and system performance Access to data holdings have limited on-line disk capability Manual vs. automated operations (ex. Load balancing) CLASS is addressing these constraints though several initiatives and standard software releases that will be discussed as part of the enterprise discussion

30 System Evolution Ongoing Initiatives 30

31 CLASS Projected Archive Volumes 31

32 32 NOAA Enterprise Archive System (CLASS) Cumulative Total Volume by Data Type

33 CLASS Projected Ingest and Delivery 33

34 CLASS Evolution to an Enterprise Archival System Evolve the existing CLASS hardware and software infrastructure into a distributed, modular, service-oriented architecture Allow greater flexibility in supporting, to the maximum extent possible, not only large data arrays from satellite programs but also provide additional archival storage services for all of NOAA’s environmental data that has been approved for archive. Working with the NOAA National Data Centers, the new Enterprise Archival Storage architecture will consist of: –Generic ingest services for flexible data acquisition, –Flexible access services using existing community standard, open-source and emerging technologies such as cloud services, –Standardized metadata repository to support a variety of search and discovery services and –Long-term, secure archival storage and data management capabilities......continued… 34

35 CLASS Evolution to an Enterprise Archival System(con’t) Preservation Planning Administration Ingest Access Data Management Archival Storage Archival Storage PRODUCERPRODUCER CONSUMERCONSUMER requests results Present CLASS Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Future CLASS (in green) Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface CLASS provides components of the OAIS-RM as services and interfaces. Additional systems and services implemented by the data centers. 35

36 Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface Ongoing Initiatives Leading toward the Enterprise Archival System Receipt Node/ Gateway Performs Data Integrity Checks Provides Temporary Data Storage Provide translation of data if necessary Provides a static XML schema for passing the elements to CLASS Assigns UUID for preservation in CLASS Passes metadata information to search and discovery services CLASS pivot points Pre-Ingest Services (Receipt Node/Gateway) Common Access Interface (M2M) Core System Upgrades Data Managers’ Toolkit Rules Base Middleware Common Storage Service (Cloud Pilot Project) Common Interface Definition 36

37 Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface Ongoing Initiatives Leading toward the Enterprise Archival System M2M (Common Access API) Search, order, and holdings information RESTful, Asynchronous, interaction with external systems Returns all data submitted via Common Ingest Interface Metadata publishing to data center catalogs CLASS pivot points Common Ingest Interface (Gateway) Common Access Interface (M2M) Core System Upgrades Data Managers’ Toolkit Rules Base Middleware Common Storage Service (Cloud Pilot Project) Common Interface Definition 37

38 Ongoing Initiatives Leading toward the Enterprise Archival System Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface Common Storage Service Access area for newly arrived and often used data sets Single interface for all access systems Push once, read many model Utilizes Cloud Services and Cloud technologies in hybrid model Extends CLASS cache out to the Enterprise CLASS pivot points Common Ingest Interface (Gateway) Common Access Interface (M2M) Core System Upgrades Data Managers’ Toolkit Rules Base Middleware Common Storage Service (Cloud Pilot Project) Common Interface Definition 38

39 Ongoing Initiatives Leading toward the Enterprise Archival System Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface Hardware/OS refresh All CLASS software migrated to Linux on new hardware Significant improvements in performance HPSS migration will provide more reliability and flexibility in configuration options CLASS pivot points Common Ingest Interface (Gateway) Common Access Interface (M2M) Core System Upgrades Data Managers’ Toolkit Rules Base Middleware Common Storage Service (Cloud Pilot Project) Common Interface Definition 39

40 Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface Ongoing Initiatives Leading toward the Enterprise Archival System Common Administration Interface Single interface data managers supports: Stewardship tools Metadata updating and versioning Data holdings monitoring and statistics CLASS pivot points Common Ingest Interface (Gateway) Common Access Interface (M2M) Core System Upgrades Data Managers’ Toolkit Rules Base Middleware Common Storage Service (Cloud Pilot Project) Common Interface Definition 40

41 Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface Ongoing Initiatives Leading toward the Enterprise Archival System Rules based middleware Data Transport Moves data objects between systems Data Information Sharing Synchronizes metadata between systems Holds data location information Orchestration Implements Enterprise process flow and routes products through Enterprise Controlled by data managers and stewards CLASS pivot points Common Ingest Interface (Gateway) Common Access Interface (M2M) Core System Upgrades Data Managers’ Toolkit Rules Base Middleware Common Storage Service (Cloud Pilot Project) Common Interface Definition 41

42 Satellite NOAA Enterprise Archival Storage System NOAA Enterprise Archival Storage System Ingest Interface Diss. Interface Diss. Interface Access Interface Administration and Preservation Climate. Gov Ordering Discovery Commercial Cloud Services Dissemination Services Access Services Model Insitu Pre-ingest Services Data Gateway Private Cloud Services Web Accessible Folders Direct Delivery Admin. Interface Ongoing Initiatives Leading toward the Enterprise Archival System Common Interface Definition Supports interaction between the components in the OAIS-RM Granules Collections Browse Images Static and well documented Common Set of Elements Common Schema Unique Identifiers CLASS pivot points Common Ingest Interface (Gateway) Common Access Interface (M2M) Core System Upgrades Data Managers’ Toolkit Rules Base Middleware Common Storage Service (Cloud Pilot Project) Common Interface Definition 42

43 Evolution Roadmap (preliminary) FY2012FY2013FY2014FY2015FY2016 NODC NCDC NGDC Phase IPhase IIPhase III Cloud Pilot Access Path Metadata Data Net IRODS Concurrent CLASS Initiatives Archive Path Data Center Migration NPP GCOM-W Jason On-Hold Programs NCDC NGDC NODC Access Dissemination Stewardship StagingStaging Archive Storage Service JPSS GOES-R M2M HPSS MOB 43

44 CLASS Evolution to an Enterprise Archival System CLASS will evolve to: Implement well-defined interfaces based on industry-standard protocols and best practices. Reduce the need for costly custom software development and allow the flexibility to use a multitude of different COTS discovery tools such as rules-based middleware for data sharing, search and discovery. Be scalable to support growth in volume and is extensible Add functionality and services as new technologies mature and enterprise needs surface. Provide a significant and rapid return on investment for NOAA and the Nation Enable the efficient and inexpensive archive of many NOAA products that are currently awaiting archival services Enables the Data Centers to provide new services and products to their customers.. Allow the efficient leveraging of other elements of the NOAA Enterprise Ground system, processing centers, distribution capabilities through generic services and interfaces 44

45 Other Initiatives that Intersect with CLASS 45

46 NOAA Enterprise Archival System, Fairmont, WV NSOF, Suitland MD NOAA Science Network(N-WAVE) Users Development Team Fairmont, WV Functions Operations Ingest Storage (Disk & Tape) Public Access Test and Integration Environment Satellite Receipt Node/ Gateway Development Environment Direct Connectivity to: Environmental Satellite Processing Center( ESPC) Product Distribution and Access (PDA) National Ice Center NOAA Coastal Watch JPSS Interface Data Processing Segment (IDPS) GOES-R Product Distribution (PD) Notional Fairmont Consolidation Plan System Configuration Off-site Backup Facility Direct Connectivity to: Environmental Satellite Processing Center( ESPC) Product Distribution and Access (PDA) Backup JPSS Interface Data Processing Segment (IDPS) Backup GOES-R Product Distribution (PD) Backup Data Centers, NOAA Gateways Commercial Cloud Services 46 Fairmont Consolidation Approach Phase I – Migrate Development, Test and Integration Environments Phase II- Establish backup Ingest Capabilities for JPSS, GOES-R, PDA Phase III- Migrate NGDC assets to Fairmont Phase IV- Consolidate NCDC assets into Fairmont Federal Data Center Consolidation

47 NESDIS Data Center Consolidation Implementation team established Begin in 2015 Budget? Complete by 2023? Three Data Centers into One? Final Organizational Structure Plan not yet approved, in preliminary planning phases but is a popular idea within NOAA 47

48 NESDIS Enterprise Ground System NOAA/NESDIS assembling EGS to meet its mission Planning phases Technical Reference Model PDA feasibility Study Common Storage Various approaches under consideration include archive, dissemination Driving question- Does CLASS to become the Enterprise Archive part of the EGS? 48

49 CLASS Programmatic Challenges Support for the Revision and Rebaseline of Level 1 Requirements –Analysis of Alternatives (Exhibit 300) –Update Full Lifecycle Cost Estimate (GAO Report) Clear program governance structure Adequate Funding for Operations Rebalance of Requirements/Cost/ Schedule for GOES-R and JPSS –Budget profile currently does not align with requirements or delivery schedule –Funding Demarcation –Where does the Program’s funding responsibility end? –Post Launch Support –Operations –Access 49

50 CLASS Technical Challenges MetaData- Need to define common elements and format (ISO, ECHO, Etc.) for ingest and access Interoperability- Date Information Exchange Language ( can include meta-data and data formats) for ingest and access Catalog- Search and discovery services outside of CLASS domain (Catalog Services for the Web) Data harvester Storage- External storage for dissemination and access outside the CLASS domain Con Ops- CLASS has a different role for satellite data and insitu data 50

51 Overlap with DAARWG Issues Priority Order from a CLASS Prospective : 1. What to archive (Not CLASS but has significant implications) – What to Archive needs to be coupled with the Budget to Archive 2. Data format (integration and interoperability)- netCDF4 3. Metadata- Elements and Schema 4. Access – Storage – Cloud for access (and dissemination) – Access (They do NOT mention Catalogs – maybe that is GEO-IDE???? 5. GEO-IDE – Is it still an active viable program 51


Download ppt "Comprehensive Large Array-data Stewardship System Status Presented to DAARWG Kern Witcher CLASS Program Manager November 8, 2012."

Similar presentations


Ads by Google