1. Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Current Status  Framework is in place and algorithms are being integrated  Satellite Data Ingested:  MODIS  SEVIRI  Simulated ABI  GOES  Integrated Algorithms  Algorithms (mostly ABI algorithms)  Cloud Mask  Cloud Phase  Cloud Height  Cloud Type  Cloud Top Temperature  Cloud Top Pressure  Cloud Optical and Microphysical Properties  Land Surface Temperature  NDVI  Fire  Legacy Sounding Products  Total Precipitable Water  Stability Indices  SST  Aerosol Optical Depth  Aerosol Particle Size  Smoke and Dust Detection  Ozone  Derived Motion Winds  Hurricane Intensity  Snow Cover  Volcanic Ash  Lightning Detection (GLM)  Rain Rate  Downward Shortwave Radiation: Surface  Upward Shortwave Radiation: Surface  Upward Longwave Radiation: TOA  Upward Longwave Radiation: Surface  Downward Longwave Radiation: Surface  Cloud and Moisture Imagery  Current Work  Installation of the following algorithms:  Snow Depth  Sea and Lake Ice Cover/Concentration/Extent/Age/Motion  SO2 Detection  Low Cloud and Fog  Rainfall Potential  Rainfall Probability  Ocean Currents and Offshore Currents  Aircraft Icing Threat  Surface Albedo  Green Vegetation Fraction  Flood/Standing Water  Surface Emissivity  Convective Initiation  Enhance-V/Overshooting Top  Visibility  Absorbed Shortwave Radiation: Surface  Turbulence Ancillary Data Product Precedence Tree Abstract NOAA/NESDIS/STAR has designed, developed, and implemented the GOES-R Algorithm Working Group (AWG) Product Processing System Framework. The framework enabled the development and testing of the Level 2 Advance Baseline Imager (ABI) and the GOES- R Lightning Mapper products within a single system. Thirty five GOES-R ABI algorithms have been run within the framework with product precedence. In integrating and running these algorithms, a number of issues arose. Some of the issues between algorithms include: different forward models, different interpolation of model forecast data, identifying ancillary data sets that provide similar data and choosing one for all algorithms to use, different missing values, multiple error handling functions, different algorithm interfaces, passing data structures between C++ and Fortran90, inter algorithm dependencies, and multiple data formats for input and output data. Each of these issues has been addressed during the development of the framework and the implementation of the algorithms. The approach to solving these issues and the resulting solutions will be discussed. Requirement: Overall observing systems architecture design Science: What improvements to observing systems, analysis approaches, and models will allow us to better analyze and predict the atmosphere, ocean, and hydrological land processes. Benefit: Improve Weather Forecasting and Reduce Loss of Life from Disasters AWG Product Precedence Tree with Refresh Rate Product Precedence  Products may require other products, ancillary data sets and/or product information from previous time periods (temporal data)  The Framework understands the precedence for each algorithm  When the inputs are read in, the product information is stored  Once the product information is stored, then the software sorts the precedence information  This sorted precedence information is used to define the calling sequence for each product  If a precedence is not run, then the algorithm is run using default information (climatology)  Ancillary data is treated as a dependency allowing common data sets used by multiple algorithms to be loaded only once by the system to avoid redundancy Product Algorithm Interface with Framework  For each algorithm, created a common interface that is used by the AIT Framework and the algorithm research system  Common interface enables a plug and play capability between the research system and the AIT Framework:  All data required by the algorithm from the Framework is passed through this interface  Reduces time that it takes to roll back the algorithm changes to the product teams due to integration into the Framework  Reduces integration time for future deliveries  Initial Design  Data structures were passed into the algorithm interface function  Depending upon the algorithm, pointers from the algorithm variables to the data structures were set up either in the algorithm or in the interface function  Significant work on integrating algorithm into the Framework  If pointers are set up within the algorithm, then the pointers would have to be reset for future deliveries  No reduction of software, each integration is different  Upgraded Design  Algorithm interface is redesigned to only pass a “Context” to the algorithm  The “Context” consists of the Framework data structures  Algorithm access the individual variable data though subroutine calls by passing the “Context”  Advantages:  Algorithm does not have to conform to the Framework data structures  Algorithm research software can have their own set of data access subroutines which are replaced by (or have the same name as) Framework subroutines  One set of Framework data structure access subroutines for all algorithms  Algorithm does not need to know anything about how the data is prepared for the algorithms use  Customize options for data interpolation verses using nearest point data  Algorithm specific I/O is accessed through subroutines – algorithm becomes I/O independent  The data access subroutines can be used to retrieve data for different instruments Framework Details  GOES-R AWG Product Processing System Framework – a stand along program to run all GOES-R Level 2+ products  The Framework is used to integrate and test all AWG Level-2 algorithms  Designed to handle large volumes of data where all the radiance and ancillary data is stored in memory; this decreases processing time by reducing I/O  Framework written in C/C++ and has fully integrated Fortran90 interfaces  Data is passed between C++ and Fortran using data structures that are synchronized between the languages  Run products in order determined by the production rules  Common data structures and software libraries.  Algorithms have been developed using common ancillary data for consistent science across products  Common radiative transfer model – the radiative transfer calculations are computed once  One data format (NetCDF4) for all inputs and outputs  Common software libraries  Enables the evaluation of the algorithms for scientific accuracy in a controlled environment Science Challenges: Modifying the 57 algorithms to use the same radiative transfer forward model and ancillary data sets while meeting the functional and performance specifications using the full product precedence tree. Next Steps: Implementing the final 22 algorithms. Transition Path: Algorithms are delivered to the Geostationary Operational Environmental Satellite series R (GOES-R) Project Office to produce the operational GOES-R products. GOES-R AWG Product Processing System Framework: Implementing Algorithms Walter Wolf 1(GOVERNMENT PRINCIPAL INVESTIGATOR), Z. Cheng 2, S. Sampson 3, R. Garcia 4, G. Fu 2, G. Martin 4, Q. Guo 3, W. Straka 4, P. Keehn 2, S. Qiu 2, E. Schiffer 4 and M. Goldberg 1 1 NOAA/NESDIS/STAR, 2 PSGS, 3 IMSG, 4 CIMSS