A Possible Suite of Algorithms for the Retrieval of Water Vapor Using Saphir/Megha-Tropiques Filipe Aires, Laboratoire de Météorologie Dynamique, IPSL/CNRS 3 rd ISRO-CNES Joint Workshop, Ahmedabad, October 2005

Goal III Develop inversion algorithm for Megha-Tropiques - Emphasis on water vapour profile with SAPHIR - Other variables… - Information fusion for multiple instrument retrievals II Tools to analyse the observing system - Realistic information content of satellite observations - Impact of First Guess: forecast versus climatological FG I Databases generation - First Guess database -”Learning database” to calibrate retrieval algorithms (NN or Bayesian)

Coincident level-1c Satellite Observations SaphirMadrasMSG FLAGPrecip / Deep convection - Clear - Cloudy: (high, medium, low) PRODTBs-Total WV -Liquid Water -TBs T cloud top Ts Climatological Dataset ECMWF 6h forecast Pattern Recognition Temp & WV profiles First guess Temp & WV Neural Network or Bayesian Case Management: - Clear - thin clous (high, medium, lower) - Rainy - Ocean / Land A priori Temp & WV Level-2a Diurnal Cycle Ana./Interpol. Gridded Temp & WV Level-2b

Outline of the Presentation Preliminary Results on First Guess and Training Databases for Megha-Tropiques Illustration with Previous Remote Sensing Applications: - High-dimension/noisy observations: atmospheric profiles with IASI - First guess information: temperature and WV over land with AMSU - Multiple-wavelength & uncertainty assessment: LST and MW emissivities over land with SSM/I

DataBase Sampling PCA Clustering RTTOVS ERA40 Reanalyses: - temperature - wv - ozone - skt Extracted Prototypes = weather regimes Prototypes in TB space TB Observations PCA/Pattern Recognition First Guess: - temperature - wv - ozone - skt

Mean atmospheric profiles Clear  Thin clouds  Convective clouds Latitude: ± 35º Ocean / Land Multivariate: - Temperature profile - Water vapour profile - Ozone profile (not necessary) - any ERA40 variables(SKT, TCWV,…) - any auxiliary dataset (ISCCP for cloud properties) Sampling on ERA40 / 6-hourly / 1999 Some filters: - rh > 1% - elevation < 500m - tcc < 50% - large scale and convective precip = 0 First Guess Dataset / Clear Case

Covariance and correlation matrices  25 PCA components Strong correlation structure  impor- tance of the PCA

PCA Base Functions TemperatureWater vapour  Anomalies with respect to mean profiles

Database Generator using Clustering Algorithm: K-means using Mahalanobis distance (i.e. using PCA)  statistical sampling, but could be uniform with another sampling technique Number of prototypes: only 10 first, to analyze the results, check that the choices are satisfactory (variables, distance, sampling algorithm, filters, case management, etc.) Next stage: thousands of prototypes extracted  FG and learning datasets Then, inversion algorithms: Bayesian and NN

Extracted Prototypes (weather regimes) Variability mostly conducted by water vapor Some variability from temperature, mostly under 300 hPa

Extracted Prototypes Dry Atm.Wet Atm.

SKT and TCWV Distributions for each Prototype SKT Histograms TCWV Histograms

Seasonality of Prototypes January 1999July 1999  Most frequent weather regime in the month

Map of Most Frequent Prototype Clusters over OceanClusters over Land  Structures are coherent (ITCZ, …) wich implies that the technique is pertinent and that the choices that have been made are correct.  Confidence for the databases generated

Prototypes Population We could emphasize extreme events or some particular regimes

Clear  Thin clouds  Convective clouds Latitude: ± 35º Ocean / Land Multivariate: - ERA40 Temperature profile - ERA40 Water vapour profile - ERA40 Cloud type, cloud amount, LWP… - ISCCP for cloud statistics: random sampling First Guess Dataset / Partly Cloudy Case Alternative: Use Meso-scale model simulations such as Meso-NH

Database Generator: perspectives Next: - Thousands of prototypes extracted  FG & learning datasets - Then, inversion algorithms: Bayesian and NN - Thin clouds / convective clouds Questions: - Sampling in space of geophysical variables (i.e. what we are interested in) or in the space of TBs (i.e. what is being observed) - Sampling statistically (learning database) or uniformly (FG database) - Distance for rare / particular events - Additional variables (especially for cloudy datasets)

Illustration with Previous Remote Sensing Applications: - High-dimension/noisy observations: atmospheric profiles with IASI - First guess information: temperature and WV over land with AMSU - Multiple-wavelength & uncertainty assessment: LST and MW emissivities over land with SSM/I

IASI Instrument High-Resolution Interferometer in the Infrared CNES / Eumetsat - Metop - in flight in 2005 Missions: operational meteorology, climatology, atmospheric chemistry Goal: 3-D description of the atmosphere and surface geophysical variables Instrument characteristics: - spectral domain: 15.5 to 3.62  m (645 to 2760 cm -1 ) - spectral resolution power: 0.25 cm -1 (more than 8400 channels) - spatial resolution: 9 km pixels - instrument noise: Gaussian Retrieval specifications: - atmospheric temperature profile: 1K RMS, 1 km - atmospheric water vapor profile: 10%, 1-2 km - atmospheric ozone profile: 10% RMS, 2 or 3 measures - surface temperature: 0.5K RMS Infrared Atmospheric Sounding Interferometer METOP1 SATELLITE Collaboration: - A. Chedin - N. Scott

Mean IASI Spectrum and Mean IASI Noise Characteristics Aires, Chedin, Scott, and Rossow, JAM, 2002.

Compression of the IASI Spectra using PCA Each spectrum X is decomposed in a base of “Eigen-Spectrum” X i X = c 1.X 1 + c 2.X 2 + … + c n.X n The number of components used n is lower than size of observations

De-Noising of the IASI Spectra using PCA Each spectrum X is decomposed in a base of “Eigen-Spectrum” X i X = c 1.X 1 + c 2.X 2 + … + c n.X n +  Suppressing high-order components suppresses noise 

A Sample of the Retrieval of a Wet Situation

Statistics of the Retrieval for Wet Situations Aires, Rossow, Scott, and Chedin, JGR, 2003a and 2003b.

Sample of the Retrieval of a Dry Situation

Statistics of the Retrieval for Dry Situations

AMSU Atmospheric Humidity and Temperature Profiles Over Land From AMSU-A and AMSU-B Observations (1) - First Guess information - ISCCP: cloudy / clear, skin temperature - ECMWF temperature and humidity profiles 6 hours before the AMSU observation - AMSU surface emissivities (2,3) at 23.8, 31.4, 50.3, 89, and 150 GHz - Observations: Observed brightness temperatures at AMSU frequencies NN - Temperature profile - Humidity profile + Simulated noise Advanced Microwave Sounding Unit (1) Karbou, F., Aires, F., Prigent, C. Retrieval of temperature and water vapor atmospheric profiles over Africa using AMSU microwave observations. Journal of Geophysical Research, 110(D7), (2) Prigent, Rossow, Matthews, Global maps of microwave land surface emissivities: Potential for land surface characterization, Radio Science, 33, (3) Karbou, Prigent, Eymard, and Pardo, Microwave land emissivity calculations using AMSU-A and AMSU-B measurements, IEEE TGRS, 43(5), Over land Collaboration: - C. Prigent - F. Karbou - L. Eymard

Impact of FG for Temperature Profile

Impact FG for Specific Humidity Profile

Bias Statistics / FG and NN Retrieval

RMS Statistics / FG and NN Retrieval

From the Visible to the Microwave: VIS and N-IR: NOAA / AVHRR visible (  m) and near-infrared reflectances (  m) Thermal IR: NOAA / AVHRR and geostationary (Meteosat, Goes E and W, GMS) thermal infrared observations (~12  m) Passive microwaves: DMSP / SSM/I passive microwave data (between 19 and 85 GHz i.e. between 3.53 mm and 1.58 cm ) Active microwaves: ERS scatterometer (5.25 GHz i.e cm) Collaboration: - C. Prigent

Visible and Near-IR (NOAA/AVHRR) (NDVI) Thermal IR (ISCCP) (diurnal Ts amplitude ) Passive microwave (DMSP / SSM/I) (surface emissivities) Active microwave (ERS scatterometer) (backscattering coefficient) Example of monthly mean products for each wavelength range - significant processing involved - many more products available

NOAA + GEOST. (VIS & IR) ISCCP Algorithm Neural Net. Retrieval Kohonen Clustering SnowWetlandsVegetation Soil Moisture Ts Diurnal Cycle Ampl. Spline / PCA Interpolation Neural Net. Retrieval Cloudy Clear ERS (MW active) AVHRR (VIS &NIR) SSM/I (MW passive) Land surf. Reflectances Backscatering Coefficients MW model Pre- Processing Pathfinder Calibrated Satellite Observations Land Surf. Emissivities Surf. Skin Temperature NCEP Reanalysis

SURFACE SKIN TEMPERATURE Ts - Energy and water exchanges at the land -surface boundary largely controlled by the Ts and Tair difference - No routine measurements of Ts - Thermal infrared provides estimates under clear sky conditions only International Satellite Cloud Climatology Project (Rossow and Schiffer, BAMS, 1999) - Development of a method to retrieve Ts under clouds using combined SSM/I microwave and IR satellite measurements (Aires et al., JGR, 2001; Prigent et al., JAM, 2003) - Systematic calculation of surface temperature from combined microwave and IR for an all-weather time record (10 years of data soon processed)

SURFACE SKIN TEMPERATURE Ts Learning data base Learning phase Operational phase Microwave and IR analysis, using a neural network with a priori information

Theoretical errors for Ts calculated on data base: no bias related to surface emissivities or cloud cover No in situ measurements available for Ts validation: comparaison with Tair at 2m. Check for expected Ts -Tair variations with solar zenith angle, surface humidity, cloud cover. (Prigent et al., JGR, 2003) SURFACE SKIN TEMPERATURE Ts

Retrieval Results Aires, Prigent and Rossow, JGR, Prigent, Aires and Rossow, JAM, Prigent, Aires and Rossow, JGR, 2003.

Statistical analysis of the available Ts previous estimates to infer the full Ts diurnal cycle over continents. Use of PCA representation and iterative optimization technique (Aires, Prigent, and Rossow, JGR, 2004) Realistic diurnal cycles are derived, for use in land-atmosphere models SURFACE SKIN TEMPERATURE Ts

Uncertainty Assessment NN: C 0 = C in + G t. H -1. G Var. Ass.: C x = (S a -1 + K t. S y -1. K) -1 Local Gaussian approximation around retrieval (first-order of error) Similar to variational estimations Clean statistical estimation, no spatial information provided Can specify all sources of uncertainty if available a priori, otherwise, distinguish only the constant and non-constant term Uncertainty is increased by outliers / incoherencies but for that, needs info on interactions (i.e. correlations) provided by G Can estimate uncertainty of very complex quantities (i.e. Jacobians) using Monte-Carlo if needed, in high-dimensional spaces Aires, Prigent and Rossow, JGR, 2004a, b, and c.

Estimation of Retrieval Uncertainties STD Error for WVSTD Error for Ts STD Error for 19V STD Error for 19H

Impact of Input Incoherencies on the Retrieval Uncertainty

Impact of Spectral Incoherencies on the Retrieval Uncertainty

Impact of Individual Perturbations on the Retrieval Uncertainty

CONCLUSIONS FOR NN RETRIEVALS Global approach for remote sensing of past and next-generation instruments: compression, de-noising, first guess, inversion algorithm, uncertainty, multi-wavelength, analysis of results (NN Jacobians) First guess information is essential: can measure contribution of a priori & obs. - provides more information - regularize the inverse problem with additional constraint Capitalize on the nonlinear correlation structures: - among inputs (satellite observations, first guess) - among outputs (temperature, water vapor, surface emissivities, etc.) - between inputs and outputs (inverse of the RTM) Merging of satellite data very powerful: - helps separate contributions of the various parameters - benefits from the complementarity between observations - more robust to noise or missing data in one type of observation Analysis tool for the results Potential synergy with variational assimilation schemes Need for databases of consistent / coherent, well-documented, reliable in situ / satellite measurements

Coincident level-1c Satellite Observations SaphirMadrasMSG FLAGPrecip / Deep convection - Clear - Cloudy: (high, medium, low) PRODTBs-Total WV -Liquid Water T cloud top Ts Climatological Dataset ECMWF 6h forecast Pattern Recognition Temp & WV profiles First guess Temp & WV Neural Network or Bayesian Case Management: - Clear - Partly cloudy (h, m, l) - Rainy - Ocean / Land / viewing angle A priori Temp & WV Level-2a Diurnal Cycle Ana./Interpol. Gridded Temp & WV Level-2b

CONCLUSIONS Need for RTTOVS Radiative Transfert Model for Madras/Saphir and at the same time for METOP & NPOES Need to have a set of radiosondes for validation purpose Our thorough analysis of the observing system should allow us to answer to de some important configuration/questions: - information from MSG or not? - temperature profile from ECMWF forecasts? - what should be done for cloudy situations - link with precipitation algorithm (Madras)