1. SERVICE ORIENTED ATMOSPHERIC RADIANCES (SOAR): A SYSTEM OF SERVICES DELIVERING MULTI-SENSOR GRIDDED DATA RECORDS AND APPLIED SCIENTIFIC ANALYSES

2. Overview Rationale for SOAR as SOS Architecture SOAR system walk through Examples of climate analysis services Summary

3. What is SOAR? S ervice O riented A tmospheric R adiances A web-based service oriented framework to improve access and use of satellite acquired atmospheric IR radiance data Maps multi-sensor radiance data from orbital coordinates stored at distributed archives onto lat-lon grids of arbitrary resolution on- demand Serves up fundamental data records in image and/or structured data formats Provides observation-based analysis services for the study and intercomparison of long historical IR data records Currently serves up AIRS and MODIS data and selected periods of HIRS3, VTPR and AVHRR3

4. What makes SOAR a system of services? SOAR includes a general ’Downloader’ for accessing multi-sensor instrument data from distributed archives from multiple satellites SOAR contains a generalized ‘Gridder’ that incorporates the sensor’s spatial response function to accurately calibrate the emitted radiance from each observed footprints portion of a grid cell SOAR is based on an SOA architecture that employs a cloud computing paradigm for efficient data intensive processing SOAR provides a set of analysis tools application specific to the geometry of the respective instrument data sets

5. Maps level 1B granules into gridded level 3 radiances Increases accessibility and usability by scientists Most op’n’l forecast centers assimilate gridded radiances Provides analysis services based directly on observations Greatly reduces data volume (lossy compression) A common framework for gridding multi instruments No level 3 gridded radiance products available from AIRS and MODIS or HIRS/AVHRR instrument teams Why we need SOAR?

6. SOAR Web Service Architecture Web Service Lookup Publish Services WSDL URLS Web Service Provider Web Service Client SOAP UDDI Directory HTML HTTP SOAR Client Server User (Browser) SOAR Web Process Services File Share (Binary Data/Images) Bluegrit Supercomputer Cluster GDAAC MODAPS NCDC NCAR

7. SOAR Downloader A C++ based application for each sensor Downloading formats tailored to each sensors data files as a separate service on UDDI Sensor access invoked by soap message specified through client server pull down window Accesses on-line data sets over internet using protocols such as ftp, http, etc.

8. AIRS Visible Image (1 day)

9. Download of HIRS 3 Orbit

10. SOARs Generic Gridder Philosophy: Philosophy: -- Common gridding algorithms for many instruments. -- Common gridding algorithms for many instruments. – Spatial calibration with recursive ray casting algorithm – Spectral response functions for calibrating multi-sensors – Orbital drift calibrations with neural network algorithm. Framework developed for gridding radiances from many Framework developed for gridding radiances from many scanning instruments. scanning instruments. – Currently implemented for AIRS, MODIS, AIRS Visible -- Extending to HIRS2/3, HIRS4 and VTPR – Incorporating artificial neural network algorithm into Gridder.

11. Cloud Computing SOAR System of Services SOAR Web System System Services Query & Request Instrument Data Transactions recorded in database Analysis Routines NCAR Graphics Visualization Real-time Data Gridding Subset by resolution (1°x 0.5° native) Subset by geographic region Remote Data Acquisition Downloader Engine Gridder Engine Data Visualization Engine Analysis Routines Data Query Web Application Transactions Pre-Gridded data AIRSMODISHIRS Staged Gridded Radiance Data MODIS gridded data (Nov’04 – Oct’08) AIRS gridded data (Sep’02 – Oct’08) Placing new datasets into production: HIRS, VTPR, SIRS, AVHRR, SBUV, OMI AVHRRVTPR

12. One Month 1/1/2005-1/31/2005 AIRS VIS Ch1

13. HIRS3 Ch 8 (12um) AIRS 528 ch 528 (12um) Jan. 1-14, 2005

14. SOAR Home Page

15. SOAR Request Page

16. SOAR Results Page

18. Some Analysis Routines Monthly, Seasonal, Yearly Averages and Anomalies Tracking monthly shifts in Inter Tropical Convergence Zone Madden Julian Oscillations (EEOFs) EL Nino Southern Oscillation (ENSO) Outgoing Long Wave Radiation (OLR) and Latitudinal OLR AIRS-MODIS Intercomparisons by Grid cell or Region Quasi-Bienneial Oscillation

19. AIRS Monthly average 0.5 o x1 o at 12.18 µm

20. AIRS Monthly anomaly 0.5 o x1 o at 12.18 µm Year to year variances Cold radiances Feb 05 (strong El Nino year) convective cloud, Warm radiances Feb 07 cloud clear surface in Western Pacific Similar in Indian Ocean and West Pacific area Feb 05 warmer than other 2 year in East US (hurricanes) Variances in Intertropical Convergence Zone Feb 2006 anomaly Feb 2005 anomalyFeb 2007 anomaly

21. MJO- results C) Variances color code BT color code Variances color code Dec1506 Dec1706 Dec1906 Dec2106 Dec2306 Dec2506 Dec2706 Dec2906 Jan0107 Jan0307 Jan0507 Jan0707 Jan0907 Jan1107 Jan1307 Jan1507 Jan1707 lag1 lag2 lag3 lag4 lag5 explaining about 14.3% variance The first EOF explaining about 14.3% variance B) 2 day running mean of MODIS channel 32 (Surface/Cloud Temperature) at 0.5 o x1 11.7 µm -12.2 µm 5S-5N 0-180E Brightness Temperature descending orbit from Dec 15 2006 to Jan 17 2007. A) The extended EOF captures the dynamics using a temporal lag of 2 day running mean.

22. AIRS/MODIS total OLR 0.5 o x1 o AIRS/MODIS total OLR isentropic assumption Compare with CERES/ERBE OLR Feb. 2005 vs April 1985

23. Summary SOAR SOS provides transparent access, gridding and visualization on demand of the following multi- sensor IR radiance data sets: AIRS, MODIS, AIRS VIS, HIRS3, VTPR; SOAR incorporates a variety of analysis tools that provide multi- year monthly and seasonal anomalies, MJO, ENSO, OLR and statistics within a grid cell as well as regional and global; SOAR system provides multiple gridding options for arbitrary spatial/spectral resolutions for multi sensor intercomparisons; SOAR Scientific Findings:  AIRS and MODIS IR spectral radiance measurements have not degraded in over 6 years  AIRS and MODIS gridded IR spectral radiances have potential to provide long term (>10 year) Fundamental Decadal Data Record  MJO can be tracked directly from raw observations with fewer uncertainties

24. Back Up Slides

25. SOAR Service Interaction Diagram User ClientWeb ServiceBluegrit New Request Submit New Request Form Get radiance data() :sessionKey login() Submit login Session Key Get login page Welcome Page/Recent Results New Request Form Request Status :requestID Login Page (HTML) Result List Get user results() :sessionKey Science Image File Request Status Page :requestID Get Results(requested) :sessionKey Get raw data Raw Data File Handle Subset/Average Data Condensed Data File Handle Render Data as Image Get Request Results Request Results File URL(s) Set Status Results Display Image/Animation/Data URL(s)

26. What is SOAR? A web based system with an interface for accessing and invoking gridding and analysis services on level 1B Infra-red radiance data Employs SOA technologies to discover and select services for use of multi -sensor infra-red radiances Serves up pre-gridded (lat-lon) AIRS and MODIS IR spectral radiances on-demand and requested image and/or structured data formats Provides a platform for users to exploit IR data for climate analysis with traditional methodologies

27. Gridding MODIS/AIRS in SOAR system (SOAR- Service Oriented Atmospheric Radiances) NASA GSFC archived Servers Processor Server (Bluegrit) 10Gbps Schedule jobs Gridding routines Bluegrit, IBM Blade Center JS20 Blades & JS21 bladeshttp://bluegrit.cs.umbc.edu/soar/ Web Server (Bluegrit) Requests display Gridding routines Subset, images Simple statistic tools Convolution routines Climate applications Sensor’s datasets requests visualize analysis download

28. SOAR Technologies Used Apache Tomcat – application server Apache – web server C/C++ – data processing utilities Java – application programming Apache AXIS – SOAP protocol library PHP – web client programming PostGRESQL – application database Apache ANT – build and deployment scripting Subversion – configuration management