A Generalized Approach to Microwave Satellite Data Assimilation Quality Control and Preprocessing Kevin Garrett 1 and Sid Boukabara 2 11 th JCSDA Science Workshop on Satellite Data Assimilation June 6, 2013 College Park, MD 1: NOAA/NESDIS/STAR, JCSDA 2: NOAA/NESDIS/STAR, JCSDA

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Outline Introduction to the MIIDAPS 1DVAR retrieval process Potential applications to NWP (GSI) Current progress Next steps 2 Goal: To have a flexible, consistent algorithm applicable to a variety of sensors for use as a preprocessor to NWP data assimilation systems which: provides quality control information about radiance observations provides dynamic information about scenes (precip, surface conditions) has consistent error characteristics across all sensors (using retrieved parameters) is flexible and easily extended to new/future sensors is mindful of computing resources/overhead/latency has meaningful, positive impact on analyses and forecasts

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Assimilation/Retrieval  All parameters retrieved simultaneously  Valid globally over all surface types  Valid in all weather conditions  Retrieved parameters depend on information content from sensor frequencies MIIDAPS Overview Multi-Instrument Inversion and Data Assimilation Preprocessing System 3 MIIDAPS S-NPP ATMS DMSP F16 SSMI/S DMSP F17 SSMI/S DMSP F18 SSMI/S GPM GMI MetOp-A AMSU/MHS MetOp-B AMSU/MHS GCOM-W1 AMSR2 Megha-Tropiques SAPHIR/MADRAS TRMM TMI NOAA-18 AMSU/MHS NOAA-19 AMSU/MHS Inversion Process  Consistent algorithm across all sensors  Uses CRTM for forward and jacobian operators  Use forecast, fast regression or climatology as first guess/background MIIDAPS 1DVAR is based on the Microwave Integrated Retrieval System (MiRS)

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD MIIDAPS Overview 4 Over All Surfaces Using All Channels -90° 90° 0°0° Latitude 170° 100 Layers Temperature EmissivitySkin Temperature Core state variables (products) from MIIDAPS Example of MIIDAPS retrieval using S-NPP ATMS with vertical cross sections at 170° longitude. Water Vapor Rain & Graupel Cloud Post Processing TPWRain Rate

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Obs Error [E] No Convergence 1DVAR Retrieval/Assimilation Process 5 Initial State Vector [X] Climatology Forecast Retrieval mode Assimilation mode CRTM Simulated TBs Observed TBs (processed) Compare Convergence Solution [X] Reached Compute  X K Update State Vector [X] Iterative Processes Covariance Matrix [B] Bias Correction 2. Retrieval done in reduced (EOF) space Reduce the dimensionality of the covariance matrix from 400x400 to 22x22 (or less depending on sensor) Transform [K] and [B] to EOF space for minimization 1. Solution is found by minimizing the cost function: Convergence is determined by non-constrained cost function: 3. X is updated through the Levenberg-Marquardt equation:

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD 1DVAR Retrieval/Assimilation Process 6 1 st Attempt2 nd Attempt Temperature Water Vapor Cloud Liquid WaterRain Water Profile Ice Water Profile Skin Temperature Surface Emissivity State Vector Parameters per Attempt MIIDAPS allows a maximum of 2 retrieval attempts per observation – 1 st attempt assumes no scattering signal in the TBs – 2 nd attempt assumes scattering from rain/ice is present in TBs – Maximum of 7 iterations per attempt Tunable parameters: nattempts, niterations, channels used (optimize efficiency without degrading outputs) Chi-square with out scattering Chi-square with scattering

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD 1DVAR Retrieval/Assimilation Process 7 Surface preclassifier determines which background and covariances to initialize for retrieval (left) Retrieved 23 GHz EmissivityRetrieved TPW Over All SurfacesIn All Weather Seamless transition along surface boundaries Emissivity inclusion in the state vector is vital for retrieval/assimilation Emissivity sensitivity to rainfall rate for AMSU-A frequencies Retrieved Rainfall Rate error as a function of retrieved emissivity error

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Applications to NWP 8 Primary objective for a 1DVAR preprocessor on microwave observations Chi-square based QC Cloudy/rainy radiance detection Emissivity constraint/ assimilation Focus for Global NWP using GSI Use chi-sq for QC/filtering Use CLW/RWP/GWP for detecting cloudy/rainy obs For filtering or assimilation Non-precip cloud/precipitating cloud Use surface emissivity as boundary condition for forward simulations to increase surface channel observations Chi-square based QC Cloudy/rainy radiance detection Emissivity constraint/ assimilation

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Applications to NWP 9 Retrieved TPW Emissivity vital for assimilation of surface sensitive channels in all weather NEXRAD 5/4 5/8 5/10 Average emissivity spectra before/after a 3-day rain event in May /4-5/8 shows ~8% change in 23 GHz emissivity. 5/5-5/7

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Applications to NWP 10 Primary objective for a 1DVAR preprocessor on microwave observations Temperature 400 mb TPW Rainfall Rate Focus for Regional NWP using GSI +HWRF Assimilation of sounding data near tropical storm cores Assimilation of TPW Assimilation of rainfall rate retrievals Temperature 400 mb TPW Rainfall Rate

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Applications to NWP 11 ‘read_sensor’ routines “setuprad” (clw, O-B filtering) Implementation of the 1DVAR preprocessor Implementation of 1DVAR preprocessing at the Bufferization stage: Process all radiance observations during time window CPU time spent outside of assimilation (minimize effect on latency) Encode 1DVAR output in BUFR as metadata or in unique BUFR file Increased control for radiance thinning/selection during GSI read process Maintain ability to use 1DVAR geophysical outputs on optimized set Implementation of 1DVAR preprocessing BUFR read stage: Process all radiance observations during time window Increased control for radiance thinning/selection during GSI read process Maintain ability to use 1DVAR geophysical outputs on optimized set Separate 1DVAR interface for each satellite sensor Read routines must be parallelized CPU time added to the analysis (how much can be afforded?) Implementation of 1DVAR in “setuprad” stage: Process only on thinned set of observations Maintain ability to use 1DVAR geophysical outputs on optimized set CPU time used in analysis (how much can be afforded?) Code is universal for all satellite datasets (single interface to 1DVAR)

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Current Status Testing currently underway with implementation in read_atms routine – 1DVAR called for additional QC (based on chisq) – No optimized thinning implemented (every 5 FOVs/Scanlines) Prelimenary implementation in setuprad routine – Still testing the interface 12 Current operationalWith additional 1DVAR filter

11 th JCSDA Workshop on Satellite Data Assimilation - College Park, MD Future Work Continue with implementation both in setuprad and in the read routines for optimized thinning Test impact of cloud filters, use of emissivity in number of obs, O-B, O-A, etc. Extend to other sensors, starting with infrared Apply to regional HWRF (product assimilation) Involve other interested JCSDA partners (NCEP, OAR, GMAO, Navy, AFWA, NCAR) 13