1. CERES FM-5 Delta Design Review 1 CERES FM-5 NPP Science Processing Delta Design Review GSFC, MD August 26, 2008 Erika Geier Jim Closs Denise Cooper Sunny Sun-Mack Lee Bodden Michael Little NASA Langley Research Center

2. CERES FM-5 Delta Design Review 2 Agenda CERES Introduction Operations Concept Design Assumptions Science Data Processing Requirements Instrument Subsystem Design Clouds Subsystem Design Testing Strategy ASDC Implementation Schedule Risk Analysis Issues/Concerns

3. CERES FM-5 Delta Design Review 3 CERES Mission Ground Systems NASA Langley Research Center (LaRC) provides critical cloud and Earth radiation budget climate data records to support global climate change research Past ERBE and current CERES instruments on TRMM (1), Aqua (2), and Terra (2), have provided 24 years of sustained radiation budget measurements LaRC has collected over 30 Instrument years of CERES data from TRMM, Terra, and Aqua and has processed over 90% of that data to date

4. CERES FM-5 Delta Design Review 4 CERES Current Capabilities Extensive LaRC CERES experience accumulated through the support of the five CERES instruments currently in orbit Existing scientific expertise for the development, calibration, management and application of CERES data products Data Center Operations already in place providing ingest, archival, production, management and distribution of CERES data products Existing Science Computing Facility (SCF) for scientific analysis, investigations and development of CERES data products Highly-experienced staff already in place supporting: –Ongoing algorithm investigation, development and enhancements –Implementation of algorithms into CERES Data Management System –Enhancements and development to current and new CERES data production systems –Production and distribution of CERES data products in a configured operations The CERES DM task was successfully appraised at CMMI Capability Level 3 /Equivalent Maturity Level 2

5. CERES FM-5 Delta Design Review 5 Science CERES Climate Analysis Research System (CARS) Organization Data Management Team (DMT) Atmospheric Sciences Data Center (ASDC) Implements algorithms Maintains software Verifies data Assists in validation Provides CM and documentation support Ingests data Places operational software in production Produces data sets Distributes data sets Archives data Provides User Services Derives & refines algorithms Validates algorithms Validates CERES data sets Writes Quality Summary Algorithm DevelopmentAlgorithm Implementation Data Production

6. CERES FM-5 Delta Design Review 6 Required CERES CARS Functionality Provide Land PEATE sub-sampling software for aggregated VIIRS SDRs (Science and DMT) Acknowledge data from SDS Land PEATE (ASDC) Ingest data from SDS Land PEATE & other sources (ASDC) Produce, archive, distribute data products (ASDC) Report status of collection (ASDC) Support user access to collection (ASDC)

7. CERES FM-5 Delta Design Review 7 CERES Operations Concept CERES instrument raw data transmitted to the C3S and IDPS IDPS creates CERES Raw Data Records (RDRs) and provides RDRs to the SDS SD3E and to ADS/CLASS SD3E provides CERES RDRs and VIIRS xDRs to the Land PEATE Land PEATE passes CERES RDRs along to ASDC Land PEATE runs VIIRS Sub-sampler; provides output to ASDC ASDC ingests and archives inputs coming from Land PEATE ASDC generates and archives higher-level CERES products ASDC distributes products to science researchers and public

8. CERES FM-5 Delta Design Review 8 CERES Operational Data Flow SD3EL-PEATE ADS IDPS SDS All RDRs All xDRs CERES RDRs, VIIRS Sub-samples & aerosols C3S Commands, Loads and requests RDRs SDRs EDRs IPs CERES Instrument Ops Team Svalbard Cmd & Tlm Mission Data Existing data, agreements Data Providers Mission Notices and Data RDRs to PST CERES CARS Data and Science Operations Mission Notices and Data requests Data Distribution Data Users Reuse existing systems and interfaces System enhancements for NPP CERES Reuse existing systems and interfaces System enhancements for NPP CERES NPOESS Systems NPP Systems NOAA Systems CERES Systems

9. CERES FM-5 Delta Design Review 9 CERES Data Processing Design Assumptions CERES CARS data processing development and operations leverages the existing personnel, procedures, production systems, and infrastructure already in place supporting CERES instruments on EOS Terra and EOS Aqua Missions CERES CARS will use existing interfaces at the LaRC ASDC CERES CARS software development is extensively reusing existing Terra/Aqua processing codes Primary required NPP related enhancements are: –Use CERES raw data in new Raw Data Record (RDR) format:  Develop a preprocessor to distribute/sort RDR contents into Aqua-like files that can be used as inputs to existing CERES processing stream –Use VIIRS data products:  Use VIIRS sub-sampled calibrated radiance and aerosol products in place of instantaneous MODIS data.

10. CERES FM-5 Delta Design Review 10 NOAA IDPS Assumptions Acquire CCSDS Packets HDF Wrap CCSDS Packets Generate & Stage RDRs for all APIDs for SDS & CLASS Maintain CDFCB Provide samples of each CERES RDR Provide 28 hours of CERES RDRs for testing CERES codes at LaRC

11. CERES FM-5 Delta Design Review 11 SDS/Land PEATE Assumptions Subscribe to SD3E for all CERES RDRs (Science, Diagnostic, and Housekeeping/Telemetry) Acquire CERES RDRs and forward to ASDC via existing Land PEATE interface Support CERES reordering of missing data Produce aggregated VIIRS SDRs (MODIS HDF-EOS format) in 5 minute granules (VIAE, VMAE, VDAE). Sub- sample and ship 5 minute granules (VIMD) to ASDC Produce VIIRS aerosol in 5 minute granules (VAOT) and ship to ASDC Post-launch: When VIIRS calibration and/or algorithm changes, reprocess VIIRS from covers-open forward and ship to ASDC to support CDRs

12. CERES FM-5 Delta Design Review 12 New Requirement Received, Logged, and Analyzed Requirement Accepted and Assigned Subsystem Updates and Unit Testing Updates Integrated Into Baseline Subsystem Science Testing Pre-CM Testing Delivery to CM CM Testing Release to SIT Operational Testing at ASDC ValRx Testing Requirements Management Subsystem Product Integration and Verification CM Validation and Product Integration ASDC Validation Software Development Process Software Development Process is in the following document currently under CERES Management review: Clouds and the Earth’s Radiant Energy System (CERES) National Polar-orbiting Operational Environmental Satellite System (NPOESS) Preparatory Project (NPP) Data Management System (DMS) Software Management Plan Version 0

13. CERES FM-5 Delta Design Review 13 Science Data Processing Approach for CERES FM-5 Processing is divided into 2 Streams: Instrument and ERBE-like –Autonomous stream, dependent only on CERES data –In production within 48 hours of power-on –Aids in verifying Instrument health –Critical for establishing CERES calibration/validation –Produces ERBE-like data sets  Data sets exist for CERES instruments on TRMM, Terra, and Aqua  Similar data sets exist for ERBE instruments on ERBS, NOAA-9 and NOAA-10 Fused data sets beginning with Cloud/Convolution/Inversion processing and the SSF –Use inputs from imager and other data sources  Higher resolution imager data is Point Spread Function (PSF) weighted –Produces higher quality climate data  Data sets exist for CERES instruments on TRMM, Terra, and Aqua –Will not be in production immediately after covers open  Requirement is to ingest VIIRS radiance, geolocation, and aerosol data when it becomes available from Land PEATE

14. CERES FM-5 Delta Design Review 14 Simplified CERES Processing Flow SS4 Compute Clouds, Convolve with CERES, compare TOA/surface fluxes CERES RDRs SSI Instrument SS2 Instantaneous ERBE-like SS3 ERBE-like TISA ES-4 ES-9 BDSES-8 IES Autonomous Subsystems affected by changing NPP inputs Autonomous CERES Subsystems VIIRS Radiances Geolocation Aerosols SS9 TISA Gridding SS10 TISA Averaging SRBAVGSSFSFC SS5 Compute fluxes at levels SS6 TISA Gridding SS7 & 8 TISA Averaging AVG SYN CRSFSW ZAVG Subsystems affected by NPP input

15. CERES FM-5 Delta Design Review 15 Current CERES Climate Data Record Production Architecture

16. CERES FM-5 Delta Design Review 16 CERES Input Data Type of DataParameter Description FreqSourceComments CERES RDRsInstrument level 0 data, attitude, and ephemeris ~131/dayLand PEATEExpecting 3 types of RDRs: Science, Telemetry, and Diagnostic VIIRS L1B Calibrated Radiance Data Imager radiances & Geolocation Aerosols 288/day ~144/day Land PEATEHave identified radiance subsets and will provide associated code Aerosol dataAerosol (Coln) Optical thickness, type/size 1/dayMODAPSFor Terra/Aqua using MODIS MOD08 and MATCH. Plan to do same for NPP Ozone dataOzone Profile1/day SMOBA 1/day OMI NCEP SMOBA OMI Meteorological data3-D Met Data 2-D atmospheric data 2-D constants 4/day 24/day 1 GMAO Precipitable Water2-D constants2/dayGlobal Hydrology Resource Center (GHRC) Geostationary dataMCIDAS data from 5 geostationary satellites per month 120/dayUniversity of Wisconsin Space Science and Engineering Center (SSEC) Only every 3rd hour is used for production SURFMAP(Snow/Ice)Snow/Ice Map4/dayNCEP/NESDIS SURFMAP(Snow/Ice)Snow/Ice Map1/dayNSIDC Existing Data Source

17. CERES FM-5 Delta Design Review 17 CERES Publicly Available Output Products Data ProductData Product NameFile Freq BDSBidirectional Scan1/day ES-8ERBE-Like Inst TOA Filtered Radiances1/day ES-9ERBE-Like Monthly Regional Averages1/mo ES-4ERBE-like Monthly Geographical Averages1/mo SSFSingle Scanner Footprint TOA/Surface Fluxes & Clouds1/hr SFCMonthly Gridded Radiative Fluxes & Clouds36/mo SRBAVGMonthly TOA/Surface Averages5/mo CRSClouds and Radiative Swath1/hr FSWMonthly Gridded Radiative Fluxes & Clouds60/mo SYNSynoptic Radiative Fluxes and Clouds1/day AVGMonthly Regional Radiative Fluxes and Clouds1/mo ZAVGMonthly Zonal and Global Radiative Fluxes & Clouds1/mo

18. CERES FM-5 Delta Design Review 18 Data Production Scenarios CERES strives to receive all possible FM-5 level 0 data and VIIRS data Instrument –Ingest manager at ASDC to monitor missing RDR files and request those files from Land PEATE –Land PEATE has agreed to provide CERES 99% of RDR files produced by IDPS within 1 month –Data is actually processed three times  Baseline1-QC run ~ 6 hours after end of the day. Used for quick instrument assessment. Files not needed after Edition1-CV processes.  Edition1-CV run ~5-20 days after the end of the month. Assume all available data has been ingested. Calibration/Validation data set used to compute gains and spectral response functions.  Edition2 run 6-12 months later using Edition1-CV as input. Applies best available instrument gains and spectral response functions. Cloud/Convolution/Inversion processing of the SSF –Ingest manager at ASDC monitors missing MODIS files and requests missing files from MODAPS. VIIRS files to be handled in similar fashion. –Once in routine production, Land PEATE has agreed to provide CERES 99% of available VIIRS files within 2 months of data date. –Requires CERES and VIIRS radiance/geolocation data to avoid data gaps in SSF files  If either one missing, a data gap results  VIIRS aerosol data also needed. If unavailable, aerosol parameters set to CERES default fill values and no data gap results. –Current Terra and Aqua SSFs typically run 6-12 months after data date. NPP processing will be very similar to Terra and Aqua.

19. CERES FM-5 Delta Design Review 19 CERES CARS Requirements Management CERES Level 3 and Level 4 requirements are being baselined in a CM-controlled document CERES Level 3 requirements have been reviewed by the SDS and will be provided to the SDS for inclusion in the SDS Requirements Specification Requirements change requires approval by Project Management and the Configuration Control Board Requirements are mapped into subsystem(s) for implementation Requirements are mapped into tests for verification and validation

20. CERES FM-5 Delta Design Review 20 Land PEATE CERES CARS CERES RDRs VIIRS Geo/Rad VIIRS Aerosols VIIRS Subsample code CERES RDRs Sub-sampling VIIRS Geo/Rad VIIRS Aerosols Instrument processing ERBE-like processing Clouds processing Archive, Distribution User Community CERES Data Sources SD3E 1- The Land PEATE receives CERES RDRs and VIIRS xDRs from the SD3E. 2- CERES RDRs are passed directly to the CERES CARS, where they are processed by the Instrument subsystem to produce inputs to the ERBE-like and Cloud subsystems. 3- The Land PEATE produces MODIS-like VIIRS radiance/geolocation and aerosol files. 4- The Land PEATE sub-samples the MODIS-like VIIRS radiance/geolocation data using software provided by CERES. 5- The Land PEATE sends the sub-sampled VIIRS files and the MODIS-like VIIRS aerosol files to the CERES CARS. 6- The CERES CARS runs the downstream subsystems, beginning with Clouds, as data becomes available. 7- Climate-quality data products are archived and made available to the user community along with Data Quality Summaries and user support.

21. CERES FM-5 Delta Design Review 21 CERES SDS Requirements NPP CERES FM-5 Science Data Segment Requirements: The ASDC Ingest subsystem shall have the capability of ingesting, verify by Checksum, and archiving CERES RDRs, sub-sampled VIIRS radiance/geolocation files, and VIIRS aerosol data received from the Land PEATE. The CERES CARS Preprocessor of the Instrument Subsystem shall have the capability of producing level-0, attitude, and ephemeris files as expected by the processing software. The CERES DMT shall have the capability of generating and delivering to the ASDC the following Science Subsystem code: - Instrument (at launch) - ERBE-like (at launch) - Clouds and subsequent subsystem software (12+ months after launch) The CERES DMT shall provide software to the Land PEATE to sub- sample VIIRS radiance/geolocation files, and work with the Land PEATE to ensure that the sub-sampling code executes properly.

22. CERES FM-5 Delta Design Review 22 CERES SDS Requirements NPP CERES FM-5 Science Data Segment Requirements (cont.): The CERES DMT shall have the capability of modifying and redelivering production code for all 12 CERES Subsystems to produce CERES NPP data products. The ASDC shall have the capability of generating, archiving, and disseminating climate-quality data products. The CERES DMT and Science Team shall have the capability of validating and writing quality summaries for all CERES data products. The ASDC shall have the capability of providing quality summaries and user support for data customers.

23. CERES FM-5 Delta Design Review 23 Requirements Verification 1.Baseline LaRC FM-5 L4 requirements by placing CERES CARS Requirements Specification Document on NX (CERES CARS, 09/2008) 2.Baseline SDS FM-5 L3 requirements by adding CERES to SDS Requirements Specification Document (SDS, 09/2008) 3.Map FM-5 L3 requirements to SDS Testing Scorecard and schedule (Vic Buczkowski, 10/2008) 4.Produce Requirements Traceability Matrix to map LaRC L4 requirements to CERES CARS Subsystems and test cases (CERES CARS, 10/2008) 5.Conduct FM-5 SDS L3 testing as documented in SDS Testing Scorecard (SDS/LaRC Test Team, (02/2009) 6.Conduct FM-5 LaRC L4 testing as documented in LaRC Requirements Traceability Matrix (LaRC Test Team, (mid-2009)

24. CERES FM-5 Delta Design Review 24 Instrument Subsystem Design Approach Instrument Design Approach and CERES DMT RDR Tests Denise Cooper Contributing Team Members: Ashley Alford Dale Walikainen Mark Timcoe

25. CERES FM-5 Delta Design Review 25 Instrument Design Approach Extensive reuse of existing software for instrument subsystem –Proven system used for current CERES processing –Majority of software written in Ada Newly developed code to be in C++ –RDR Preprocessor  Creates 24-hr Level-0 and Toolkit compliant Ephemeris and Attitude data files Ada code will be updated to allow processing of FM-5 data PGE to process incoming NPP Level-0 data will be the first priority, other PGEs will be converted following successful testing of this PGE.

26. CERES FM-5 Delta Design Review 26 Instrument Design Approach PGE providing data to ERBE-like already in C, only changes will be to allow processing of FM-5 data. Initial version of all code will use the SDP Toolkit due to schedule constraints. Updates to remove dependency on the SDP Toolkit expected after launch. CERESlib updates to create a C-version of existing F90 Metadata creation routines. –Facilitate integration of existing Metadata with Ada, C and C++ software.

27. CERES FM-5 Delta Design Review 27 NPP Instrument Subsystem CERES RDRs L0 Data Coeff Data Geolocate & Calibrate BDSs IESs Pre-Es8 Generator Pre-ES8 To Convolution To ERBE-Like Attitude Data Preprocessor ~133 Science & Diagnostic RDRs with Diary data 2 files needed to process (time span noon yesterday - noon today) 14 files needed to process (time span hr 22 day before - hr 00 next day) Up to 3 files per day (Science, Calibration & Diagnostic) Up to 24 files per day (HDF4 Vdata containing Geolocation values Filtered radiances) Up to 6 files per day BDS, BDSS, BDSD, BDSM, BDSP & BDSG (HDF4 SDS & Vdatas containing Geolocation values Raw data counts Filtered radiances Instrument H&S info) 1 file per day (Binary containing Geolocated, Filtered radiances) Ephemeris Data Attitude Data Attitude Data Attitude Data Attitude Data

28. CERES FM-5 Delta Design Review 28 Spectral Response Function Modtran Data Unfiltering Coefficients Unfiltering ERBE-like Spectral Corrections No code changes needed to support NPP

29. CERES FM-5 Delta Design Review 29 Daily Inversion Pres8 EID6 ES4 ES9 ES8 Monthly Averaging ERBE-like Inversion From Instrument Subsystem No code changes needed to support NPP Unfiltering Coefficients 1 file per day (Binary containing Geolocated, Filtered radiances) 1 file per day (HDF-EOS4 containing Geolocated Filtered & Unfiltered radiances & fluxes) 1 file per day (Binary database Info) 1 file per month (HDF4 containing Regional, Zonal & Global Gridded fluxes) 1 file per month (HDF4 containing Global Average fluxes)

30. CERES FM-5 Delta Design Review 30 CERES DMT RDR Tests Objectives –Ensure the RDR preprocessor is able to read the CERES Science and Diagnostic RDR HDF5 files and create the properly formatted Level-0, Ephemeris and Attitude data files that are compatible with the updated Ada and the converted C++ code. Description –Assume ingest of RDRs by ASDC; Alternative is to access RDRs stored in specified location if not available from the ASDC –Test with selected RDRs on the development platform to provide output that will be used by the Level-0 data processor to show that output Level- 0, Ephemeris and Attitude data is as expected by this PGE. –Test on the development platform with minimum of 28-hrs of RDR data through the Level-0 PGE, on to the Pre-ES8 generator to create the data used by ERBE-like.

31. CERES FM-5 Delta Design Review 31 CERES DMT RDR Tests Description (Cont’d) –Pre-delivery testing on production platform to ensure all systems work on the production platform and generate scientifically equivalent data.  Ensures all software works as expected on the production platform before delivery  Provides expected output used by CM to verify their test results –CM testing using Instrument Subsystem Test Plan to verify that all necessary updates have been delivered and provide output equivalent to the provided expected output. –ASDC SIT operational testing: e.g. Dec. 31, Jan. 1, Feb. 28/29, etc. Participants –Ashley Alford, Denise Cooper, Mark Timcoe, Land PEATE, LaRC ASDC SIT

32. CERES FM-5 Delta Design Review 32 CERES DMT RDR Tests RDRs Science & Diagnostic ~133 per data day (Level-0 packets & Diary data) RDR Pre-processor Science Level-0 File Calibration Level-0 File Diagnostic Level-0 File Total 10391 packets Per data day Ephemeris Data Files Attitude Data Files Data files (Noon previous day - Noon next day) 12 Data files (2 hr data per file)

33. CERES FM-5 Delta Design Review 33 Instrument Schedule RDR Format Documentation Provided (Jun. 2008) Unpacking of RDR Wrapper Implemented (Jul. 2008) Conversion of RDR Contents to CERES Level 0 Format Implemented (Sep. 2008) Selection and Conversion of Attitude/Ephemeris Data Implemented (Sep. 2008) Informal Testing with Actual RDR (dependent on IDPS providing early RDR) (Oct. 2008) Instrument Subsystem Enhancements Integrated and Tested (Dec. 2008) Instrument Subsystem Verification Testing Complete (Jan. 2009) Delivery to ASDC through DMT CM for Formal Validation Tests (by Mar. 13, 2009)

34. CERES FM-5 Delta Design Review 34 Clouds Subsystem Design Approach CERES Clouds Subsystem Design Approach and CERES DMT VIIRS Tests Sunny Sun-Mack Contributing Team Members: Yan Chen Walt Miller

35. CERES FM-5 Delta Design Review 35 CERES SSF Processing (Subsystem 4) Subsystem 4 - Determine Cloud Properties, TOA and Surface Fluxes, is made up of 3 different sets of code Clouds - VIIRS inputs will necessitate changes to support NPP –Processes the imager data –Resulting pixel-level output written to temporary file Convolution - minor changes needed to support NPP –Merges the pixel-level data with the CERES footprint data –Resulting footprint-level output written to a temporary file Inversion - no change needed to support NPP –Reads the footprint level output file containing CERES radiance data and cloud properties –Computes unfiltered radiances, TOA fluxes, and surface fluxes –Generates the SSF data product which is archived and publicly released

36. CERES FM-5 Delta Design Review 36 NPP Impact on CERES Clouds and Convolution Design Approach VIIRS will replace MODIS as imager source –Ability to generate certain cloud properties changes –Some radiances, CO 2 slicing, may be missing CERES IES (input from Instrument Subsystem) interface will remain the same New Point Spread Function (PSF) ancillary file will be generated to account for VIIRS imager resolution SSF interface change possible as CERES migrates to Edition3

37. CERES FM-5 Delta Design Review 37 Clouds Subsystem Design Approach Newly developed VIIRS sub-sampling code at LaRC –VIIRS Sub-sampling Preprocessor  Written in C language  Input: NPP_VIAE ( Radiances from Imaging “I” Channels ) NPP_VMAE ( Radiances and geolocations from Moderate-Resolution “M” Channels) NPP_VDNE ( Radiances and geolocations from Imaging Day-Night Band (channel) “DNB” )  Output: NPP_VIMD_SS ( Radiances and geolocations from all above three Imaging, Moderate and DNB, but sub-sampled and sub-channeled with certain channels being averaged )

38. CERES FM-5 Delta Design Review 38 Clouds Subsystem Design Approach VIIRS sub-sampling code will run at the Land PEATE –The package to be delivered to Land PEATE by CERES CM at LaRC  VIIRS sub-sampling code  Process Control Generator to produce process control file (PCF)  Input data and Expected sub-sampled output  Test Plan (PDF) –VIIRS sub-setting code will be run at Land PEATE  The output from VIIRS sub-sampling code, NPP_VIMD_SS, will be produced at Goddard Land PEATE  Land PEATE will then deliver NPP_VIMD_SS to LaRC ASDC LaRC ASDC will ingest and archive NPP_VIMD_SS NPP_VIMD_SS will be the Cloud Imagery Data (CID) as input to CERES Clouds Subsystem.

39. CERES FM-5 Delta Design Review 39 VIIRS Sub-setting Data Formats 1.Sub-setting NPP_VIAE Resolution: 375 meter Radiances from VIIRS Imaging Channels Sub-setting Channels I1 = 0.64  m I2 = 0.865  m I3 = 1.61  m I4 = 3.74  m I1 = 11.45  m I1 = 0.64  m I3 = 1.61  m I4 = 3.74  m I1 = 11.45  m NPP_VIAENPP_VIMD_SS

40. CERES FM-5 Delta Design Review 40 VIIRS Sub-setting Data Formats 1.Sub-setting NPP_VIAE ( Cont’d) M Pixels (NPP_VIMD_SS)I Pixels (NPP_VIAE) Averaging 4 Imaging Pixels to 1 Moderate Resolution Pixel

41. CERES FM-5 Delta Design Review 41 VIIRS Sub-setting Data Formats 1.Sub-setting NPP_VIAE ( Cont’d) Sub-setting Samples NPP_VIAE NPP_VIMD_SS Scan-lines x Pixels = 4608 x 6400 Scan-lines x Pixels = 2304 x 3200 Every other pair of scan-lines Every other pair of pixels

42. CERES FM-5 Delta Design Review 42 VIIRS Sub-setting Data Formats SDS Name (NPP_VIAE)DescriptionDimensionsSDS Name (NPP_VIMD_SS)Dimensions Radiance_Img_I1Radiances for I1 in moderate resolution(6144,6400)Radiance_Img_I1_Avg(1536,1600) Radiance_Img_I3Radiances for I3 in moderate resolution(6144,6400)Radiance_Img_I3_Avg(1536,1600) Radiance_Img_I4Radiances for I4 in moderate resolution(6144,6400)Radiance_Img_I4_Avg(1536,1600) Radiance_Img_I5Radiances for I5 in moderate resolution(6144,6400)Radiance_Img_I5_Avg(1536,1600) Reflectance_Img_I1Reflectance for I1 in imagery resolution(6144,6400)Reflectance_Img_I1_Sub Reflectance_Img_I1_Avg (3072,3200) (1536,1600) Reflectance_Img_I3Reflectance for I3 in imagery resolution(6144,6400)Reflectance_Img_I3_Sub Reflectance_Img_I3_Avg (3072,3200) (1536,1600) BrightTemp_Img_I4Brightness temperature for I4 at imagery resolution (6144,6400)BrightTemp_Img_I4_Sub BrightTemp_Img_I4_Avg (3072,3200) (1536,1600) BrightTemp_Img_I5Brightness temperature for I5 at imagery resolution (6144,6400)BrightTemp_Img_I5_Sub BrightTemp_Img_I5_Avg (3072,3200) (1536,1600) QF_VIIRS_I1_SDR_1Quality control flag for I1(6144,6400)QF_VIIRS_I1_SDR_1_Sub(3072,3200) QF_VIIRS_I1_SDR_3Scan quality control flag for I1(192,4)QF_VIIRS_I1_SDR_3(192,4) QF_VIIRS_I3_SDR_1Quality control flag for I3(6144,6400)QF_VIIRS_I3_SDR_1_Sub(3072,3200) QF_VIIRS_I3_SDR_3Scan quality control flag for I3(192,4)QF_VIIRS_I3_SDR_3(192,4) QF_VIIRS_I4_SDR_1Quality control flag for I4(6144,6400)QF_VIIRS_I4_SDR_1_Sub(3072,3200) QF_VIIRS_I4_SDR_3Scan quality control flag for I4(192,4)QF_VIIRS_I4_SDR_3(192,4) QF_VIIRS_I5_SDR_1Quality control flag for I56144,6400)QF_VIIRS_I5_SDR_1_Sub(3072,3200) QF_VIIRS_I5_SDR_3Scan quality control flag for I5(192,4)QF_VIIRS_I5_SDR_3(192,4) 1. Sub-setting NPP_VIAE ( Cont’d): SDSs Kept

43. CERES FM-5 Delta Design Review 43 VIIRS Sub-setting Data Formats SDS Name (NPP_VIAE) DescriptionDimensionsSDS Name (NPP_VIMD_SS) Dimensions Radiance_Img_I2Radiances for I2 in moderate resolution (6144,6400) Reflectance_Img_I2Reflectance for I2 in imagery resolution (6144,6400) QF_VIIRS_I2_SDR_1Quality control flag for I2(6144,6400) QF_VIIRS_I2_SDR_3Scan Quality control flag for I2(192,4) 1.Sub-setting NPP_VIAE ( Cont’d): The SDSs not being subset

44. CERES FM-5 Delta Design Review 44 VIIRS Sub-setting Data Formats NPP_VMAE M1 = 0.412  mM3 = 0.488  m  M2 = 0.445  mM4 = 0.555  m M5 = 0.672  mM7 = 0.865  m M6 = 0.746  mM8 = 1.24  m M10 = 1.61  mM9 = 1.378  m M12 = 3.7  mM11 = 2.25  m M13 = 4.05  mM14 = 8.55  m M15 = 10.763  m M16 = 12.013  m 2. Sub-setting NPP_VMAE Resolution: 750 meter Radiances and Geolocation from VIIRS Moderate Channels Sub-setting Channels NPP_VIMD_SS M3 = 0.488  m  M4 = 0.555  m M7 = 0.865  m M8 = 1.24  m M9 = 1.378  m M11 = 2.25  m M14 = 8.55  m M15 = 10 763  m M16 = 12.013  m

45. CERES FM-5 Delta Design Review 45 VIIRS Sub-setting Data Formats 1.Sub-setting NPP_VMAE ( Cont’d) Sub-setting Samples NPP_VMAE NPP_VIMD_SS Scan-lines x Pixels = 3072 x 3200 Scan-lines x Pixels = 1536 x 1600 Every other scan-line Every other pixel

46. CERES FM-5 Delta Design Review 46 VIIRS Sub-setting Data Formats Latitude Longitude Solar Zenith Angle Solar Azimuth Angle Sensor Zenith Angle Sensor Azimuth Angle Satellite Range Terrain Height All Geolocation Info in NPP_VMAE is included in the Subset NPP_VIMD_SS Scan Start Time Scan Mid Time NPP_VIMD_SS: SDS (64 bits float) NPP_VMAE: Global Attributes 2. Sub-setting NPP_VMAE (Cont’d) NPP_VMAE and NPP_VIMD_SS

47. CERES FM-5 Delta Design Review 47 VIIRS Sub-setting Data Formats 2. Sub-setting NPP_VMAE (Cont’d): SDSs Kept SDS NameDescriptionDimensionsSDS NameDimensions Radiance_Mod_M11Radiances for M11 in moderate resolution(3072,3200)Radiance_Mod_M11_Sub(1536,1600) Radiance_Mod_M14Radiances for M14 in moderate resolution(3072,3200)Radiance_Mod_M14_Sub(1536,1600) Radiance_Mod_M15Radiances for M15 in moderate resolution(3072,3200)Radiance_Mod_M15_Sub(1536,1600) Radiance_Mod_M16Radiances for M16 in moderate resolution(3072,3200)Radiance_Mod_M16_Sub(1536,1600) Radiance_Mod_M3Radiances for M3 in moderate resolution(3072,3200)Radiance_Mod_M3_Sub(1536,1600) Radiance_Mod_M4Radiances for M4 in moderate resolution(3072,3200)Radiance_Mod_M4_Sub(1536,1600) Radiance_Mod_M7Radiances for M7 in moderate resolution(3072,3200)Radiance_Mod_M7_Sub(1536,1600) Reflectance_Mod_M11Reflectance for M11 in moderate resolution(3072,3200)Reflectance_Mod_M11_Sub(1536,1600) Reflectance_Mod_M3Reflectance for M3 in moderate resolution(3072,3200)Reflectance_Mod_M3_Sub(1536,1600) Reflectance_Mod_M4Reflectance for M4 in moderate resolution(3072,3200)Reflectance_Mod_M4_Sub(1536,1600) Reflectance_Mod_M7Reflectance for M7 in moderate resolution(3072,3200)Reflectance_Mod_M7_Sub(1536,1600) Reflectance_Mod_M8Reflectance for M8 in moderate resolution(3072,3200)Reflectance_Mod_M8_Sub(1536,1600) Reflectance_Mod_M9Reflectance for M9 in moderate resolution(3072,3200)Reflectance_Mod_M9_Sub(1536,1600) BrightTemp_Mod_M14Brightness temp. for M14 at mod resolution(3072,3200)BrightTemp_Mod_M14_Sub(1536,1600) BrightTemp_Mod_M15Brightness temp for M15 at mod resolution(3072,3200)BrightTemp_Mod_M15_Sub(1536,1600) BrightTemp_Mod_M16Brightness temp for M16 at mod resolution(3072,3200)BrightTemp_Mod_M16_Sub(1536,1600)

48. CERES FM-5 Delta Design Review 48 VIIRS Sub-setting Data Formats SDS NameDescriptionDimensionsSDS NameDimensions QF_VIIRS_GEO_MOD_2Quality control flag for geo location(3072,3200)QF_VIIRS_GEO_MOD_2_Sub(1536,1600) QF_VIIRS_M11_SDR_1Quality control flag for M11(3072,3200)QF_VIIRS_M11_SDR_1_Sub(1536,1600) QF_VIIRS_M11_SDR_3Scan quality control flag for M11(192,4)QF_VIIRS_M11_SDR_3(192,4) QF_VIIRS_M14_SDR_1Quality control flag for M14(3072,3200)QF_VIIRS_M14_SDR_1_Sub(1536,1600) QF_VIIRS_M14_SDR_3Scan quality control flag for M14(192,4)QF_VIIRS_M14_SDR_3(192,4) QF_VIIRS_M15_SDR_1Quality control flag for M15(3072,3200)QF_VIIRS_M15_SDR_1_Sub(1536,1600) QF_VIIRS_M15_SDR_3Scan quality control flag for M15(192,4)QF_VIIRS_M15_SDR_3(192,4) QF_VIIRS_M16_SDR_1Quality control flag for M16(3072,3200)QF_VIIRS_M16_SDR_1_Sub(1536,1600) QF_VIIRS_M16_SDR_3Scan quality control flag for M16(192,4)QF_VIIRS_M16_SDR_3(192,4) QF_VIIRS_M3_SDR_1Quality control flag for M3(3072,3200)QF_VIIRS_M3_SDR_1_Sub(1536,1600) QF_VIIRS_M3_SDR_3Scan quality control flag for M3(192,4)QF_VIIRS_M3_SDR_3(192,4) QF_VIIRS_M4_SDR_1Quality control flag for M4(3072,3200)QF_VIIRS_M4_SDR_1_Sub(1536,1600) QF_VIIRS_M4_SDR_3Scan quality control flag for M4(192,4)QF_VIIRS_M4_SDR_3(192,4) QF_VIIRS_M7_SDR_1Quality control flag for M7(3072,3200)QF_VIIRS_M7_SDR_1_Sub(1536,1600) QF_VIIRS_M7_SDR_3Scan quality control flag for M7(192,4)QF_VIIRS_M7_SDR_3(192,4) QF_VIIRS_M8_SDR_1Quality control flag for M8(3072,3200)QF_VIIRS_M8_SDR_1_Sub(1536,1600) QF_VIIRS_M8_SDR_3Scan quality control flag for M8(192,4)QF_VIIRS_M8_SDR_3(192,4) QF_VIIRS_M9_SDR_1Quality control flag for M9(3072,3200)QF_VIIRS_M9_SDR_1_Sub(1536,1600) QF_VIIRS_M9_SDR_3Scan quality control flag for M9(192,4)QF_VIIRS_M9_SDR_3(192,4) 2. Sub-setting NPP_VMAE (Cont’d): SDSs Kept

49. CERES FM-5 Delta Design Review 49 VIIRS Sub-setting Data Formats 2. Sub-setting NPP_VMAE ( Cont’d): The SDSs not being subset SDS NameDescriptionDimensionsSDS NameDimensions Radiance_Mod_M1Radiances for M1 in moderate resolution(3072,3200) Radiance_Mod_M10Radiances for M10 in moderate resolution(3072,3200) Radiance_Mod_M12Radiances for M12 in moderate resolution(3072,3200) Radiance_Mod_M13Radiances for M13 in moderate resolution(3072,3200) Radiance_Mod_M2Radiances for M2 in moderate resolution(3072,3200) Radiance_Mod_M5Radiances for M5 in moderate resolution(3072,3200) Radiance_Mod_M6Radiances for M6 in moderate resolution(3072,3200) Radiance_Mod_M8Radiances for M11 in moderate resolution(3072,3200) Radiance_Mod_M9Radiances for M11 in moderate resolution(3072,3200) Reflectance_Mod_M1Reflectance for M1 in moderate resolution(3072,3200) Reflectance_Mod_M10Reflectance for M10 in moderate resolution(3072,3200) Reflectance_Mod_M2Reflectance for M2 in moderate resolution(3072,3200) Reflectance_Mod_M5Reflectance for M5 in moderate resolution(3072,3200) Reflectance_Mod_M6Reflectance for M6 in moderate resolution(3072,3200) BrightTemp_Mod_M12Brightness temperature for M12 at moderate resolution(3072,3200) BrightTemp_Mod_M13Brightness temperature for M12 at moderate resolution(3072,3200)

50. CERES FM-5 Delta Design Review 50 VIIRS Sub-setting Data Formats 2. Sub-setting NPP_VMAE ( Cont’d): The SDSs not being subset SDS NameDescriptionDimensionsSDS NameDimensions QF_VIIRS_M10_SDR_1Quality control flag for M10(3072,3200) QF_VIIRS_M10_SDR_3Scan quality control flag for M10(192,4) QF_VIIRS_M12_SDR_1Quality control flag for M12(3072,3200) QF_VIIRS_M12_SDR_3Scan quality control flag for M12(192,4) QF_VIIRS_M13_SDR_1Quality control flag for M13(3072,3200) QF_VIIRS_M13_SDR_3Scan quality control flag for M13(192,4) QF_VIIRS_M1_SDR_1Quality control flag for M1(3072,3200) QF_VIIRS_M1_SDR_3Scan quality control flag for M1(192,4) QF_VIIRS_M2_SDR_1Quality control flag for M2(3072,3200) QF_VIIRS_M2_SDR_3Scan quality control flag for M2(192,4) QF_VIIRS_M5_SDR_1Quality control flag for M5(3072,3200) QF_VIIRS_M5_SDR_3Scan quality control flag for M5(192,4) QF_VIIRS_M6_SDR_1Quality control flag for M6(3072,3200) QF_VIIRS_M6_SDR_3Scan quality control flag for M6(192,4)

51. CERES FM-5 Delta Design Review 51 VIIRS Sub-setting Data Formats DNB = 0.7  m  Latitude Longitude Solar Zenith Angle Solar Azimuth Angle Sensor Zenith Angle Sensor Azimuth Angle Satellite Range Terrain Height Subset every other scan-line and every other pixel 3. Sub-setting NPP_VDNE Resolution: 750 meter Radiances and Geolocation from VIIRS Day-Night Channel Only One channel-->No Sub-setting Channel Keep All Geolocation SDSs Sub-setting Sample

52. CERES FM-5 Delta Design Review 52 VIIRS Sub-setting Data Formats 4.File Size Summary INPUT: NPP_VIAE ---> 460 MB / granule NPP_VMAE ---> 560 MB / granule NPP_VDNE ----> 330 MB / granule OUTPUT NPP_VIMD ---> 400 MB / granule ~ 70% Data Reduction

53. CERES FM-5 Delta Design Review 53 Emissivity Maps x 4 IGBP Water % Elevation Surface Maps NPP CERES Cloud Subsystem GMAO MOA Clear Reflectance History Clouds Processor TRMM VIRS CID MODIS Terra CID MODIS Aqua CID NPP VIIRS CID Cloud Imager Data (CID) Algorithm Ancillaries Snow/ice Model BiDir Model Directional Model Angular Models EIPD (Cookie Dough) EQCG EQCB ECVS ECV Output NPP VIIRS Subset: NPP_VIMD_SS To CERES Convolution

54. CERES FM-5 Delta Design Review 54 Convolution Subsystem Description Cloud Pixel Convolution Inversion X 24 hours Instrument IES FQCI SSFAI SSFI EIPD Cookie dough FQC PSF Convolve imager radiances, cloud properties, and aerosol into CERES footprint

55. CERES FM-5 Delta Design Review 55 CERES DMT VIIRS Tests Objectives Ensure the VIIRS Sub-sampler processor is able to: (1) Read NPP_VIAE ( Imager resolution radiances), NPP_VMAE ( Moderate resolution radiances and geolocations) NPP_VDNE (Day / Night band radiances and geolocations) (2) Subset sub-channel sub-sample averaging of imagery resolution data (3) Produce VIIRS sub-sampled output NPP_VIMD_SS

56. CERES FM-5 Delta Design Review 56 CERES DMT VIIRS Tests Description –If less than a couple of hours of NPP VIIRS data is available, then Land PEATE pushes VIIRS NPP_VIAE, NPP_VMAE, and NPP_VDNE to a CERES/VIIRS designated computer –For larger volumes of data, the Land PEATE sends all test NPP VIIRS data types to the ASDC for ingest, and then subsequently staged for access by the VIIRS Sub-sampler on a CERES/VIIRS computer –Execute VIIRS Sub-sampler on local CERES/VIIRS computer to create and verify sub-sampled output using input VIIRS data from Land PEATE –Load VIIRS Sub-sampler and selected VIIRS test data in the file space and computer designated for CERES processing at the Land PEATE, and then conduct testing at the Land PEATE and verify sub-sampled output –CERES DMT delivers VIIRS Sub-sampler (tar files) and test plan to the CERES CM for delivery to the Land PEATE –Land PEATE conducts VIIRS Sub-sampler testing creating sub-sample –Land PEATE verifies sub-sample and also send sub-sampled output to either the ASDC or to a designated CERES/VIIRS computer for additional verification

57. CERES FM-5 Delta Design Review 57 CERES DMT VIIRS Tests NPP VIIRS Sub-setting Processor NPP_VIAE NPP_VDNE NPP_VMAENPP_VIMD_SS Required Input Expected Output Participants –Sunny Sun-Mack, Yan Chen, Tammy Ayers, Walt Miller, CERES Science Team members: Patrick Minnis, Norman Loeb, Tom Charlock, Dave Kratz and Dave Doelling, Land PEATE, and LaRC ASDC

58. CERES FM-5 Delta Design Review 58 CERES CARS Delivery Process to the Land PEATE Following unit, integration and verification testing by the CERES DMT, the VIIRS Sub-sampler must be delivered to the Land PEATE for final testing CERES DMT creates tar files containing source code, process control generator, selected NPP VIIRS input data and expected sub-sampled output CERES DMT updates Test Plan for testing at the Land PEATE CERES DMT delivers the above tar files along with the Test Plan to CERES CM CERES CM delivers the tar files and Test Plan to the Land PEATE’s designated machine. A “Delivery Notification” email is sent to the Land PEATE and all concerned

59. CERES FM-5 Delta Design Review 59 CERES CARS Delivery Process to the Land PEATE (cont.) Land PEATE conducts VIIRS Sub-sampler testing When the Land PEATE testing is successfully completed, the Land PEATE: –Sends an email to all concerned regarding the status –Promotes the delivery to their production environment –Makes the delivery (as tar files) available in a designated area for CERES CM to retrieve –Sends a notification email to CERES CM that this action has taken place. Upon receiving this notification, CERES CM: –Retrieves the tarred delivery –Places it in the CERES CM repository Subsequent changes to CM controlled software are documented and tracked using a CERES Software Configuration Change Request (SCCR) and redelivered through CERES CM

60. CERES FM-5 Delta Design Review 60 NPP VIIRS Sub-sampler Schedule Data Format of VIIRS Subset for CERES Defined (June 1, 2008) Completed on Time. Data Product Catalog Pages Documenting VIIRS Subset for CERES Defined (August 1, 2008) Completed on Time Sub-sampler Code Complete and Running Locally at LaRC on Linux Machine (October 1, 2008) Goddard Land PEATE Accepts the Sub-sampler Code Delivery and Produces Sub-sampled Output from Input Data (December 1, 2008) Verify a Few Days of Sub-sampled Data Produced at GSFC (January 15, 2009) ASDC Ingests Sub-sampled VIIRS Data; Verified by CERES DMT (February 26, 2009)

61. CERES FM-5 Delta Design Review 61 CERES DMT Testing Approach The CERES DM task was successfully appraised at CMMI Capability Level 3 /Equivalent Maturity Level 2 SSAI is appraised at CMMI Capability Level 3/Equivalent Maturity Level 2 CERES DMT uses a structured CMMI-based development process that includes unit and integration testing Functional (e.g., verification) tests are executed to verify requirements for all enhancements Regression tests are executed to verify existing functionality Delivery packages for CERES and VIIRS Sub-sampler to include: –Test Plan –Expected output –All inputs –Code and scripts Every CERES code delivery (to ASDC or Land PEATE) is under configuration management (CM) control Documented in CERES NPP DMS Software Management Plan Version 0

62. CERES FM-5 Delta Design Review 62 CERES CARS Testing Strategy Conversion of RDRs to Level 0 data will be tested by the DMT using defined formats and available test RDRs prior to delivery of code to the ASDC –Testing conducted in development environment Transmission of RDRs by the Land PEATE to the ASDC will be tested when: –CERES RDRs are available to the Land PEATE from the SD3E –Interface between ASDC and the Land PEATE has been established Final testing of conversion of RDRs to Level 0 data will involve: –Testing in the CERES operations test environment –Land PEATE sending all three RDR data types to the ASDC –ASDC ingesting data and acknowledging receipt to the Land PEATE –Instrument subsystem reading RDRs staged from ASDC archive and correctly converting to Level 0 format

63. CERES FM-5 Delta Design Review 63 CERES CARS Testing Strategy VIIRS Sub-sampler will be tested by the DMT using test data provided by the Land PEATE prior to code delivery to the Land PEATE –Testing conducted in development environment –Testing also conducted on designated Land PEATE computer Transmission of sub-samples by the Land PEATE will be tested when: –VIIRS Sub-sampler has been tested and delivered to the Land PEATE –VIIRS data is available to the Land PEATE from the SD3E or other source –Interface between ASDC and the Land PEATE has been established Final testing of Sub-sampler will involve: –Execution of Sub-sampler by the Land PEATE to produce sub-sampled data product –Transmission of the sub-sampled data to the ASDC for ingest and archive –CERES DMT stages data from ASDC archive and verifies sub-sample

64. CERES FM-5 Delta Design Review 64 DMT NPP Schedule 10/1/08: VIIRS sub-sampling software complete and tested at LaRC 12/1/08: VIIRS sub-sampling software running at SDS Land PEATE, expected sub-sampled output has been verified by CERES DMT 1/15/09: Large set of VIIRS sub-sampled output produced at SDS Land PEATE and verified at LaRC by CERES DMT 3/13/09: Instrument subsystem code needed to support NCT3 complete and tested by DMT 5/15/09: Instrument subsystem code delivered to ASDC, tested, and promoted to operations 08/13/09: Instrument subsystem code needed to support NCT4 complete and tested by DMT  Interim delivery to correct NCT3 problems and prepare for NCT4 10/15/09: Instrument subsystem code delivered to ASDC, tested, and promoted to operations 11/19/09: Instrument subsystem code needed to support launch complete and tested by DMT 1/15/10: Instrument subsystem code delivered to ASDC, tested, and promoted to operations

65. CERES FM-5 Delta Design Review 65 CERES CARS Build Schedule CERES Data Management System Projected Build promotion dates to ASDC Operations: Build 1: NCT3 Functionality-May 2009 Build 2: NCT4 Functionality-October 2009 Build 3: Launch-Ready System-January 2010

66. CERES FM-5 Delta Design Review 66 ASDC Implementation Approach Michael Little

67. CERES FM-5 Delta Design Review 67 ASDC Supports CERES PI in Ingest, Archive, Distribution, Production Ingesting and Archiving inputs for all Data Products –Use by Production in Producing Data Products –Use by Science Team in analysis and algorithm refinement –Long-term stewardship of data including migration across media changes  Moving from Tape archive with Disk Cache to Disk archive with Tape Backup Distribute CERES Data Products –External Customers order data products via ASDC Order Tool, ESDIS ECHO  ASDC Order Tool provides interim support  ESDIS ECHO provides access to all EOS data products  Subscription Services  Plan to supply CLASS with subscription service as an offsite backup –Internal users via file sharing  Moving from Request/deliver model to online Fiberchannel/NFS Produce Data Products for CERES PI –Accept and test Production codes from DMT –Accept Production Requests from DMT –Run codes consistently to produce expected data products  Now by procedure; expect automation within one year

68. CERES FM-5 Delta Design Review 68 ASDC Evolution Approach Provide CERES Production Services at ASDC –Meet constraints of CERES PGEs  Avoid increasing workload on DM/Science workforce  Support FORTRAN, Ada, and IDL languages  Meet increasing memory and disk access requirements –Use lessons learned to improve CERES production  Improve I/O bandwidth limitations to allow multiple instances of some subsystems  Reduce unplanned outages by increasing storage reliability  Reduce dependence on active tape archive to improve labor intensive staging processes –Increase production capacity for FM-5 and reprocessing Terra/Aqua –Replace current hardware at end-of-life Provide Access to CERES Data Products for Internal Users –Production –LaRC Science Community –CERES Data Management Team Improve Order Throughput for External Customers

69. CERES FM-5 Delta Design Review 69 64 x86 SMP 256 GB ASDC & SCF Integrated Architecture Fiberchannel Connectivity Fiberchannel Switch 64 x86 SMP 256 GB 56 P6 256GB 56 P6 256GB 56 P6 256GB 56 P6 256GB 220TB DS4800 220TB DS4800 220TB DS4800 220TB DS4800 220TB DS4800 64 x86 SMP 128 GB 16 x86 HS12 64 GB Common SANhead (NFS) (AFP) (Samba) ASDC JBOSS Server & TS-1 & Ingest ASDC SDP Production 64 x86 SMP 256 GB 56 P6 256GB SCF Cluster (Access through Sun Grid Engine) 220TB DS4800 220TB DS4800 SCF Disk Array Instrument ERBE-like SRB/Power SARB 14 4P6@8GB SCF Interactive Processors ASDC Disk Archive (R/W Production) (R/O SCF) Consodine TISA SOFA Clouds TISA 220TB DS4800 WWW 64 x86 SMP 128 GB Cloud Modeling Cluster (Xu) 64 x86 SMP 128 GB ECS DUE Processing Db Server Db Server Tesla 3755 GPU Experiment (YongHu) Sim Server FC Monitor LB1

70. CERES FM-5 Delta Design Review 70 220TB DS4800 ASDC & SCF TCP/IP Architecture LaRCNet Switch 220TB DS4800 220TB DS4800 220TB DS4800 220TB DS4800 4x16 x86 SMP 128 GB SH1 x3650 Common SANhead (NFS, AFP, Samba) ASDC JBOSS Server & TS-1 ASDC SDP Production (SGE) SCF Cluster (SGE) 220TB DS4800 220TB DS4800 SCF Disk Array (R/W) Instrument ERBE-like SRB/Power SARB 14 4P6@8GB SCF Interactive Processors ASDC Disk Archive (R/W Production)(R/O SCF) SARB TISA SOFA Clouds TISA 220TB DS4800 WWW 64 x86 SMP 128 GB Cloud Modeling Cluster 64 x86 SMP 128 GB ECS DUE Processing ANGe Db Servers ANGe Db Servers Tesla 3755 GPU Experiment Sim Server Load 2 Magneto 56 P6 256GB SH2 x3650 192.168.16.x FC Monitor ASDC SGE Head SCF SGE Head Load 1 LaRCNet 100Mbps Ethernet Private Net (DS Mgt) Private Net (GPFS) Cluster & Mgt Net (SGE) Green Indicates SubSys Function Legend WWW Proxy Servers Cluster Mon (HW TBD) 64 x86 SMP 256 GB 64 x86 SMP 256 GB 64 x86 SMP 256 GB 56 P6 256GB Zamboanga Center Firewall NISN WAN IBM Director x3650

71. CERES FM-5 Delta Design Review 71 I/F To be Resolved Metadata For Orderable Data Products ASDC Ingest Processes for NPP ANGe Drop-Box ASDC Archive ANGe Core db IMS db ASDC Ingest Processor Metadata Location & GUID Subscription Files PDR Files ASDC Delivery Server Files Local Production PDRFiles Land PEATE PDRFiles ECHO Metadata Adapter NOAA CLASS

72. CERES FM-5 Delta Design Review 72 NPP Impacts to the ASDC ImpactResolution Increased storage capacity and performance needed for new CERES data set Additional disks purchased with fiberchannel connectivity to production processors. Higher density tapes expand capacity of tape archives. Increased processing capability needed for new CERES data set Additional processors purchased and added to Linux cluster increasing processing capacity. Faster data access from new disk archive vs. tape archive. Additional CERES data production requirements effect on staffing Improvements in staging/ingest/archive reliability will offset staff workload for production. Staff experienced in processing CERES data products. Operating procedures required for new data set and experience of staff Job production using on existing Subsystem DMT Ops Manuals requiring minor updates for NPP. Staff experienced in processing CERES data products. Ability of facility to accommodate additional hardware with respect to space, HVAC and power ASDC has space/HVAC for expansion. Power upgrades 09/08/08 Increased data load impact on network throughput New architecture will improve internal data production network throughput and access by SCF users May need additional bandwidth to Land PEATE for VIIRS User access to current and future CERES data products New architecture will speed ordering CERES data products New data products will be available through current ordering tools when released to public

73. CERES FM-5 Delta Design Review 73 Ingest/Archive Sizing Estimates ASDC can accommodate expected CERES data volumes from the Land PEATE CERES RDRs: Science, Diagnostic, and Telemetry File sizes are 1370.8 KB, 685.5 KB and 27.5 KB; excluding HDF overhead, about 700 KB/file Estimate: (1370.8 + 700) KB/file * 131 files/day + 685.5 KB/file * 1 file/day+ 27.5 KB/file * 12 files/day = 272.2903 MB/day Sub-sampled VIIRS radiance and geolocation data: Output File Size: Approximately 400 MB per 5-min file Estimate: 400 MB/file * 288 files/day = 115,200.0 MB/day VIIRS Aerosol data: File Size: ~ 632 MB per 5-min file, as of June 1, 2008 57 MB per 5-min file if changing to 6 x 6 aggregation for both Land and Ocean 32 MB per 5-min file if changing to 8 x 8 aggregation for both Land and Ocean. Estimate: ~12 files per hour or ~288 files/day * 632 MB/file = 182,016 MB/day ~12 files per hour or ~288 files/day * 57 MB/file = 16,416 MB/day Note: The Land PEATE expects to switch to 6x6, or possibly 8x8, aggregation prior to launch Total Volume - Large Aerosol Files = 297,488.29 GB/day 108,583.226 GB/yr or 108.58 TB/yr Total Volume - Small Aerosol Files = 131.888.29 GB/day 48,139.226 GB/yr or 48.14 TB/yr

74. CERES FM-5 Delta Design Review 74 Security Compliance ASDC Approval to Operate Based on August, 2007 Review –System Security Plan: SC-010-M-LRC-1000  Approving Official: Steve Jurczyk, Deputy Center Director  System Owner: John Kusterer, ASDC Manager  System Security Manager: Michael Little, ASDC Systems Engineer –Re-certification in progress based on CIO mandated schedule re-alignment –Re-certification based on ANGe/IBM installation to occur next spring  Include consolidation of SCF and ASDC Systems into single System IT Security Issues –Reconciliation of OCIO Mandated ODIN conversion with ASDC ITS needs  ODIN to take over support of workstations, desktops, etc  ASDC to retain control/responsibility for servers –Implementation of unannounced ITS measures by OCIO sometimes interferes with ingest and delivery over network  Agency CIO mandated transfer of all Center Firewalls to NISN is not fully understood

75. CERES FM-5 Delta Design Review 75 CERES Production Code Test & Evaluation Proven Processes Based on 10 Years of Experience in CERES Code Delivery, Acceptance, Testing, Evaluation and Production CERES DMT Develops Code for Production of Data Products –PGE(s) for each Sub-system (SS) tested and internally verified –DMT and ASDC develop test cases –Delivery to ASDC includes sample data, Operators Manual, code –Delivery Evaluation and Code Compilation by DMT Configuration Management comparing to sample data products and using test cases –Delivery to ASDC for SS Integration and Test to evaluate producibility and completeness and comparison to sample data products –ASDC and DMT work together to identify and resolve any problems through Software Trouble Tickets –Promotion to Production status and ASDC CM when approved by Operations Readiness Review (ORR) Data Product Production Validation Testing –Once code is in production, sample data products (ValRx) are produced with appropriate key months –ValRx data products are evaluated by Science Team and approved prior to release into an orderable status  Unsuitable results may result in re-delivery –Release also requires a Quality Summary by the SS Science Lead

76. CERES FM-5 Delta Design Review 76 ASDC Validation Testing Approach Systems delivered to the ASDC undergo Validation testing in a controlled environment DMT Subsystems under CERES CM delivered to the ASDC for Validation Testing Delivered Subsystems inspected for completeness by ASDC CM, unique testing scenarios identified Subsystems validated in operations testing environment for all new requirements and enhancements Delivered subsystems regression tested for existing functionality Operations Readiness Review (ORR) following successful Validation Testing and/or mitigation of existing issues Problem reports generated and tracked for all identified problems Problem report resolutions incorporated into configured software for redelivery (if needed) to the ASDC for validation Data Production and Operations CERES CMASDC CMASDC Validation Testing ASDC CCB And Project Management ASDC Regression Testing ASDC Operations CERES FM-5 Testing to Operations Life-Cycle ProblemsResolutions

77. CERES FM-5 Delta Design Review 77 NPP-Level Testing and Status Testing of the RDRs requires the SD3E to access data from the IDPS, the RDRs are then accessed by the Land PEATE and sent to ASDC –ASDC will ingest the RDR data and acknowledge receipt –ASDC will test data production when available after 05/09 Testing of VIIRS data requires Land PEATE to run sub-sampler and send sub-sampled radiance/geolocation and aerosol data to the ASDC –Land PEATE must be able to generate or access valid VIIRS data –ASDC will ingest data and acknowledge receipt Ingest capability will be ready to support Early EEO Ingest and data production capability will be ready to support NCT3 Interim delivery to address NCT3 problems to support NCT4 success All NPP-related functionality available at L-90 days for Launch

78. CERES FM-5 Delta Design Review 78 CERES ASDC RDR Tests Objective –Verify that the Land PEATE can access CERES RDRs from the SD3E and successfully transfer the data to the ASDC for ingest and archive Description –SD3E reads CERES RDR data from IDPS or through alternative source –Land PEATE accesses RDRs from the SD3E and transfers data to the ASDC –ASDC ingests RDRs and acknowledges receipt back to Land PEATE –ASDC archives data for staging and processing by Instrument Subsystem –ASDC will test data production when systems and RDRs available after 05/09 –Additional reorder and data management functions will also be tested Input –CERES RDRs generated by IDPS or from Proxy data provided to Land PEATE Output –CERES RDRs ingested and archived at ASDC –CERES Level 0 data for instrument subsystem processing available after 05/09 Participants –ASDC, Land PEATE, SD3E, CERES DMT –IDPS when CERES RDR capability is available

79. CERES FM-5 Delta Design Review 79 CERES ASDC VIIRS Tests Objective –Verify execution of the VIIRS Sub-sampler by the Land PEATE and the transfer of the sub-sampled radiance/geolocation and aerosol data to the ASDC Description –SD3E reads VIIRS data from IDPS or through alternative source –Land PEATE must be able to generate or access valid VIIRS data –Land PEATE executes VIIRS Sub-sampler and sends sub-sample to ASDC –ASDC ingests sub-sampled data and acknowledges receipt to the Land PEATE –ASDC archives data for staging and processing by Clouds subsystem –Sub-sampled and aerosol data examined by science and DMT staff –ASDC will test data production when systems are available after 05/09 –Aerosol data ingest and archive tested when available from the Land PEATE Input –VIIRS data generated by IDPS or from Proxy data provided by Land PEATE Output –VIIRS sub-samples generated by the Land PEATE –VIIRS aerosol data product generated by the Land PEATE Participants –ASDC, Land PEATE, SD3E, CERES DMT, IDPS

80. CERES FM-5 Delta Design Review 80 Operational Support System Upgrades will Enhance Current Capabilities –CERES production system already runs at the ASDC –ANGe storage management upgrade should be operational this week –Hardware upgrades to be phased in with existing CERES Ingest, Archive, Distribution and Production capabilities in Dec09, reducing risk to NPP needs Procedures and Documentation are available and tested –Operations staff with CERES processing experience already in place with defined, proven procedures –ASDC staff already work closely with Land PEATE staff through interactions supporting MODIS data transfers provided from the same data center (MODAPS) System availability will remain at 24x7 –Current production model has minimized need for 24x7 staffing –System hardware and architecture evolution will increase system reliability and reduce workforce requirements Improved system performance and additional CERES data products will lead to increased customer satisfaction

81. CERES FM-5 Delta Design Review 81 Documentation and Agreements Most documentation and agreements already exist as part of CERES on TRMM, Terra, and Aqua –http://science.larc.nasa.gov/ceres/docs.html –Documents may be updated if needed –New documents may be written explicitly for NPP, to meet special requirements Overall roles and responsibilities are documented in CERES Science and Data Products Working Agreement –Working agreement between CERES IT and NPP Project Office –May need to be revisited when NPP transfers to NOAA leadership –Working Agreement in hands of NPP SEWG Operations Agreement (OA) being developed between ASDC and Land PEATE –Based on existing MODAPS OA for MODIS data sent to ASDC –On-going relationship between MODAPS and ASDC for MODIS No OA planned for ADS/CLASS interface –Expect to use standard customer subscription agreement

82. CERES FM-5 Delta Design Review 82 ASDC Supports NPP Schedule 08/31/08Initial Operational Capability (IOC): CERES in ANGe 09/15/08Complete Installation/Configuration of IBM hardware Sep-Nov08Test/Characterize IBM Installation 11/13/08Initial Delivery of CERES RDR to ASDC Ingest Nov08Complete LaTIS conversion into ANGe Nov/Dec08Design/Develop/Test NPP Inst, VIIRS into ANGe 11/26-12/23/08Complete CERES RDR and VIIRS ICD 12/01/08IOC IBM ASDC+SCF Archive and Processing System Jan/Feb09Debug Aerosols/VIIRS Sub-sample into ANGe with Land PEATE 05/08/09SDS Test2: Instrument RDRs into ANGe from Land PEATE 05/15/09Build 1 ORR: NCT3 Functionality 06/09-15/09NCT3 07/01/09-02/02/10Design/Develop/Test Other NPP into ANGe 10/15/09Build 2 ORR: NCT4 Functionality/NCT3 Problem Corrections 12/02-09/09SDS Test3/NCT4 01/15/10Build 3 ORR: Launch-Ready System 02/02/10Instrument/VIIRS IOC 03/02/10MRR 06/02/10NPP Launch

83. CERES FM-5 Delta Design Review 83 Overall Schedule CERES Schedule is baselined CERES Schedule incorporated into NPP Schedule Schedule available as separate handout

84. CERES FM-5 Delta Design Review 84 Resource Assessment Required resources have already been assessed and hired Current experienced CERES staff leveraged for CERES FM-5 CERES FM-5 staff augmented with key hires to address technical challenges –Major SSAI job fair resulted in hundreds of resumes Backfill positions created in current CERES support filled with new talent Resource estimate based on approximate lines of code to be generated Using industry productivity standards, CERES CARS has the resources in place needed to do the job Additional resources needed for NPP testing, processes, and working groups not included in industry standards Additional issues and requirements expected Ongoing CERES projects supported by SSAI can provide short-term support if critical needs arise

85. CERES FM-5 Delta Design Review 85 Risk Analysis Raw Data Record (RDR) Formats Not Fully Defined Given that CERES Raw Data Record (RDR) formats are not fully defined or understood in time, there is a possibility the code needed to support testing may not be complete. 21Technical Schedule Erika Geier M05/29/08 The CERES team is developing a giver/receiver list and this item is on the list. 07/23/08 IPO/Raytheon and Janet Smith all Know about this risk. Unavailability of CERES Raw Data Record (RDR) Test Data Given that CERES Raw Data Record (RDR) test data from IDPS is not available prior to testing, there is a possibility the code needed to support testing may not be complete. 31Technical Schedule Erika Geier W07/23/08 We have requested test data; although, it doesn't sound like we'll get meaningful test data. TitleDescriptionL C Impact Owner Action Comments Legend: L – Likelihood C – Consequence M – Mitigate W - Watch

86. CERES FM-5 Delta Design Review 86 CERES Issues/Concerns Need single sample of the Science, Diagnostic, Housekeeping/Telemetry RDRs to verify that we understand format no later than 10/15/08 –CERES data content may be proxy, sample, or fill data –Prefer attitude/ephemeris not be fill data Desire 3 days (~ 28 consecutive hours) of CERES RDRs to run through preprocessor and Instrument Subsystem to aid in code development no later than 11/14/08