在科研中更好的使用美国业务资料分析系统（GSI） Ming Hu   *Developmental Testbed Center, NCAR-NOAA/GSD **Cooperative Institute for Research in Environmental Sciences, Colorado University at Boulder  

Outline General introduction to the Gridpoint Statistical Interpolation (GSI) system Development Testbed Center and Community GSI History, repository, test, and review committee Available GSI resources for community O2R: website, document, tutorial, and help desk R2O: community contributions Current and future development Community GSI and operational GSI

General introduction to the Gridpoint Statistical Interpolation system This section is mainly based on John Derber’s talk in 2013 summer GSI tutorial

John C. Derber NOAA/NWS/NCEP/EMC Overview of GSI John C. Derber NOAA/NWS/NCEP/EMC

History The Spectral Statistical Interpolation (SSI) analysis system was developed at NCEP in the late 1980’s and early 1990’s. Main advantages of this system over OI systems were: All observations are used at once (much of the noise generated in OI analyses was generated by data selection) Ability to use forward models to transform from analysis variable to observations Analysis variables can be defined to simplify covariance matrix and are not tied to model variables (except need to be able to transform to model variable) The SSI system was the first operational variational analysis system system to directly use radiances

History While the SSI system was a great improvement over the prior OI system – it still had some basic short- comings Since background error was defined in spectral space – not simple to use for regional systems Diagonal spectral background error did not allow much spatial variation in the background error Not particularly well written since developed as a prototype code and then implemented operationally

History The Global Statistical Interpolation (GSI) analysis system was developed as the next generation global/regional analysis system Wan-Shu Wu, R. James Purser, David Parrish Three-Dimensional Variational Analysis with spatially Inhomogeneous Covariances. Mon. Wea. Rev., 130, 2905-2916. Based on SSI analysis system Replace spectral definition for background errors with grid point version based on recursive filters

History Used in NCEP operations for Operational at AFWA Regional Global Hurricane Real-Time Mesoscale Analysis Rapid Refresh (ESRL/GSD) Operational at AFWA GMAO collaboration Modification to fit into WRF and NCEP infrastructure Evolution to ESMF

General Comments GSI analysis code is an evolving system. Scientific advances situation dependent background errors -- hybrid new satellite data new analysis variables Improved coding Bug fixes Removal of unnecessary computations, arrays, etc. More efficient algorithms (MPI, OpenMP) Bundle structure Generalizations of code Different compute platforms Different analysis variables Different models Improved documentation Fast evolution creates difficulties for slower evolving research projects

General Comments Code is intended to be used Operationally Must satisfy coding requirements Must fit into infrastructure Must be kept as simple as possible External usage intended to: Improve external testing Reduce transition to operations work/time Reduce duplication of effort

Simplification to operational 3-D for presentation For today’s introduction, I will be talking about using the GSI for standard operational 3-D var. analysis. Many other options available or under development 4d-var hybrid assimilation observation sensitivity FOTO Additional observation types SST retrieval Detailed options Options make code more complex – difficult balance between options and simplicity

Basic analysis problem J = Jb + Jo + Jc J = (x-xb)TB-1(x-xb) + (H(x)-y0)T(E+F)-1(H(x)-y0) + JC J = Fit to background + Fit to observations + constraints x = Analysis xb = Background B = Background error covariance H = Forward model y0 = Observations E+F = R = Instrument error + Representativeness error JC = Constraint terms

Analysis variables Background errors must be defined in terms of analysis variable Streamfunction (Ψ) Unbalanced Velocity Potential (χunbalanced) Unbalanced Temperature (Tunbalanced) Unbalanced Surface Pressure (Psunbalanced) Ozone – Clouds – etc. Satellite bias correction coefficients

Analysis variables χ = χunbalanced + A Ψ T = Tunbalanced + B Ψ Ps = Psunbalanced + C Ψ Streamfunction is a key variable defining a large percentage T and Ps (especially away from equator). Contribution to χ is small except near the surface and tropopause.

Background fields Current works for following systems NCEP GFS NCEP NMM – binary and netcdf NCEP RTMA NCEP Hurricane (not using subversion version yet) GMAO global ARW – binary and netcdf – (not operationally used yet RR - GSD) FGAT (First Guess at Appropriate Time) enabled up to 100 time levels

Background Errors Three paths – more in talk by D. Kleist Isotropic/homogeneous Most common usage. Function of latitude/height Vertical and horizontal scales separable Variances can be location dependent Anisotropic/inhomogeneous Function of location /state Can be full 3-D covariances Still relatively immature Hybrid

Observations Observational data is expected to be in BUFR format (this is the international standard) Each observation type (e.g., u,v,radiance from NOAA-15 AMSU-A) is read in on a particular processor or group of processors (parallel read) Data thinning can occur in the reading step. Checks to see if data is in specified data time window and within analysis domain

Input data – Satellite currently used Regional AMSU-A NOAA-15 Channels 1-10, 12-13, 15 NOAA-18 Channels 1-8, 10-13, 15 METOP-A Channels1-6, 8-13, 15 AQUA Channels 5, 8-13 Thinned to 60km AMSU-B/MHS NOAA-15 Channels 1-3, 5 NOAA-18 Channels 1-5 METOP-A Channels 1-5 HIRS NOAA-17 Channels 2-15 METOP-A Channels 2-15 Thinned to 120km AIRS AQUA 148 Channels  IASI METOP-A 165 Channels Global all thinned to 145km   GOES-15 Sounders Channels 1-15 Individual fields of view 4 Detectors treated separately Over ocean only AMSU-A NOAA-15 Channels 1-10, 12-13, 15 NOAA-18 Channels 1-8, 10-13, 15 NOAA-19 Channels 1-7, 9-13, 15 METOP-A Channels 1-6, 8-13, 15 AQUA Channels 6, 8-13 ATMS NPP Channels 1-14,16-22 MHS NOAA-18 Channels 1-5 NOAA-19 Channels 1-5 METOP-A Channels 1-5 HIRS METOP-A Channels 2-15 AIRS AQUA 148 Channels IASI METOP-A 165 Channels

Input data – Conventional currently used Radiosondes Pibal winds Synthetic tropical cyclone winds wind profilers conventional aircraft reports ASDAR aircraft reports MDCARS aircraft reports dropsondes MODIS IR and water vapor winds GMS, JMA, METEOSAT and GOES cloud drift IR and visible winds GOES water vapor cloud top winds Surface land observations Surface ship and buoy observation SSM/I wind speeds QuikScat and ASCATwind speed and direction SSM/I and TRMM TMI precipitation estimates Doppler radial velocities VAD (NEXRAD) winds GPS precipitable water estimates GPS Radio occultation refractivity and bending angle profiles SBUV ozone profiles and OMI total ozone

Simulation of observations To use observation, must be able to simulate observation Can be simple interpolation to ob location/time Can be more complex (e.g., radiative transfer) For radiances we use CRTM Vertical resolution and model top important

Quality control External platform specific QC Some gross checking in PREPBUFR file creation Analysis QC Gross checks – specified in input data files Variational quality control Data usage specification (info files) Outer iteration structure allows data rejected (or downweighted) initially to come back in Ob error can be modified due to external QC marks Radiance QC much more complicated. Tomorrow!

Useful References Wan-Shu Wu, R. James Purser and David F. Parrish, 2002: Three-Dimensional Variational Analysis with Spatially Inhomogeneous Covariances. Monthly Weather Review, Vol. 130, No. 12, pp. 2905–2916.  R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part I: Spatially Homogeneous and Isotropic Gaussian Covariances. Monthly Weather Review, Vol. 131, No. 8, pp. 1524–1535.  R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part II: Spatially Inhomogeneous and Anisotropic General Covariances. Monthly Weather Review, Vol. 131, No. 8, pp. 1536–1548.  McNally, A.P., J.C. Derber, W.-S. Wu and B.B. Katz, 2000: The use of TOVS level-1B radiances in the NCEP SSI analysis system.  Q.J.R.M.S., 126, 689-724. Parrish, D. F. and J. C. Derber, 1992: The National Meteorological Center's spectral statistical interpolation analysis system. Mon. Wea. Rev., 120, 1747 - 1763. Derber, J. C. and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, 2287 - 2299. Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Wu, Wan-Shu; Lord, Stephen , Introduction of the GSI into the NCEP Global Data Assimilation System, Weather and Forecasting. Vol. 24, no. 6, pp. 1691-1705. Dec 2009 Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Errico, Ronald M; Yang, Runhua, Improving Incremental Balance in the GSI 3DVAR Analysis System, Monthly Weather Review [Mon. Weather Rev.]. Vol. 137, no. 3, pp. 1046-1060. Mar 2009. Kazumori, M; Liu, Q; Treadon, R; Derber, JC, Impact Study of AMSR-E Radiances in the NCEP Global Data Assimilation System Monthly Weather Review,Vol. 136, no. 2, pp. 541-559. Feb 2008. Zhu, Y; Gelaro, R, Observation Sensitivity Calculations Using the Adjoint of the Gridpoint Statistical Interpolation (GSI) Analysis System, Monthly Weather Review. Vol. 136, no. 1, pp. 335-351. Jan 2008. DTC GSI documentation (http://www.dtcenter.org/com-GSI/users/index.php)

DTC comments GSI is an operation system PrepBUFR and NCEP QC procedure Well-tested for operation applications Very strict data QC Advanced operator for radiance Bias correction is very important radiance data analysis Model top and channel selection Data coverage for regional application PrepBUFR and NCEP QC procedure The operation system supported by DTC for community users

DTC and Community GSI Who is DTC Community GSI: Goal and Building Community GSI repository Test and evaluation GSI review committee

Who is DTC ? Developmental Testbed Center (DTC) http://www.dtcenter.org/ Bridge between research and operational community NWP systems supported by DTC WRF, HWRF, GSI, MET, …

Community GSI: Goals Provide current operation GSI capability to research community (O2R) Provide a pathway for research community to contribute operation GSI (R2O) Provide a framework to enhance the collaboration from distributed GSI developers Provide rational basis to operational centers and research community for enhancement of data assimilation systems

Community GSI: Building O2R: Code portability: well-tested GSI code package for release Documentation: GSI User’s Guide GSI User’s Webpage Annual release On-site Tutorial (lectures and Practical cases) Help desk R2O: Repository: Build, maintain, sync, test Commit community contributions GSI Review Committee The important role of GSI repository and review committee

GSI Code Repository Branch GSI Trunk Tag Branch NDAS GDAS Community release HWRF Use tags or branches for: Release, new development, bug fix … Applications may use different revisions in the trunk (“snapshot”). Which GSI should I use ? There is no “DTC GSI”, “EMC GSI” or “global GSI”. There is only one GSI! For a researcher, community release should be sufficient to use. If you are interested in getting new development back to the GSI trunk, contact GSI helpdesk (gsi_help@ucar.edu) get access to the developmental version of GSI.

GSI Review Committee Branch GSI Trunk NCEP/EMC NASA/GMAO NOAA/ESRL NESDIS DTC AFWA NCAR/MMM Branch Tag GSI Review Committee Represent research and operational community Coordinate distributed GSI development Conduct GSI Code Review DTC is working as the pathway between the research community and operational community. Provide visitor program for potential contributions to the operational code. Provide equivalent-operational tests of community contributions and provide rational basis for operational implementations.

History First Community GSI tutorial Created GSI Review Committee June, 2006:North American Mesoscale (NAM) System, NCEP May, 2007: Global Forecast System (GFS), NCEP 2007: DTC started to document GSI. 2009: Created GSI code repository over NCEP and DTC First GSI release V1.0 Started user support 2010: First Community GSI tutorial Created GSI Review Committee 2011 First Community GSI workshop

Available GSI resources for community: O2R User’s Webpage Documentation Annual release package Annual residential tutorial Help desk

Community GSI – User’s Page Support mainly through User’s Page and help desk: http://www.dtcenter.org/com-GSI/users/index.php

Community GSI Release and Tutorial The GSI User’s Guide and the on-line tutorial match with official release code Summer 2013 GSI Tutorial: NCEP, Aug. 5-7, 2013

Community GSI Release Source code and fixed files were based on: the GSI EMC trunk r19180 (May 10, 2012) the community GSI trunk r826 The GSI User’s Guide and the on-line tutorial cases are updated with each official release.

Community GSI - Documents User’s Guide Matching with each official release Tutorial lectures Code browser Calling tree Publications

Community GSI- User’s Guide V3.2

Community GSI- User’s Guide V3.2

Community GSI - Practice On-line tutorial for each release Residential tutorial practice cases

Community GSI – tutorial and workshop

Community GSI - Help desk gsi_help@ucar.edu Any GSI related questions Most of questions were answered by DTC staff Forward complex questions to GSI developers

Available GSI resources for community: R2O Commit the community contributions to trunk Help desk

GSI R2O Transition Procedure (2011 Implementation) GSI Review Committee Scientific Review Development, testing and merging GSI Review Committee code and commitment review Community research Code development candidate 1 Code commitment candidate (Branches) 2 3 Code in repository trunk

GSI R2O applications DTC contributes to R2O transition, coordinate & assist code commitment: Mariusz Pagowski: Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model DTC: Portability issues from trunk head testing Mariusz Pagowski: Add WRF-Chem model file as background for PM2.5 assimilation Tom Augline: Add control and state variables for Cloudy analysis (triggered more work on the bundles of GMAO) Zhiquan Liu: aerosol work has been merged to the trunk GSD: Recent GSI developments for RR

Collaboration HFIP program: BUFR processing training AMB branch: GSI application for Rapid Refresh Mariusz Pagowski (AMB): Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model Don Birkenheuer (FAB): Thorpex work using GSI EMC GSI group (feedback from community GSI users) NCAR/MMM DA group (hybrid and cloud analysis,, chemistry analysis ) Space Weather Prediction Center /NOAA (Houjun Wang ) (whole atmosphere data assimilation using GSI) Other GSI users (OU regional ENKF, PSD (Jeff Whitaker) global ENKF …)

R2O Summary DTC also works with researchers to bring research community contributions back to the GSI operation repository (R2O) Create Trac to keep community researchers in developing loop Help researchers to review and merge the code on top of the trunk Test the code with regression test suite and initial the code review for committing Send your questions to gsi_help@ucar.edu

Current and future development Community GSI Operational GSI August 9th, 2013 Committee Meeting Agenda 8:45 GMAO 
9:30 NESDIS 
10:15 break 
10:25 ESRL 
11:10 NCAR 
11:55 lunch
12:40 DTC 
1:10 NCEP 
3:00 Adjourn

Community GSI Maintaining the current support GSI review committee Community GSI trunk: sync, test Annual release with code and updated document Improving GSI user’s Website Holding on-site tutorial Maintaining Helpdesk Help committing community contributions Application tools and new instructions to GSI (bundle, hybrid, …) Build operation-similar test system (ongoing)

Ongoing and future EMC GSI development work EMC GSI development team Update Aug. 8, 2013

Use of observations Prepare for METOP-B and CrIS implementation - Andrew Collard Implementation Aug. 20 to WCOSS Includes SEVIRI from Meteosat-10 Use of AURA MLS ozone profiles – Haixia Liu Testing of new version of real time data underway Cloudy Radiance Assimilation – Emily Liu, Yanqiu Zhu, Z. Zhang, Will McCarty (GMAO) Updates to trunk – Significant progress – Testing without convection Improved structure of radiance diagnostic files – Andrew Collard - TBD Satellite wind usage upgrade –XiuJuan Su GOES hourly wind project Ground based Doppler radar data – Shun Liu Conversion to dual-pol Aircraft based Doppler radar data – Mingjing Tong In operations on WCOSS SSMI/S data – Andrew Collard, Emily Liu, and Ellen Liang (NESDIS) To be included in Spring 2014 implementation Mesoscale surface observation – Manuel Pondeca, Steve Levine GLERL surface reduction for lakes under testing GOES-R/SEVERI radiances – Haixia Liu Additional quality control under development Channels used above low clouds CRTM – Paul VanDelst, David Groff, Quanhua Liu(NESDIS), Yong Chen(NESDIS) Improvements continue Improve use of significant levels in RAOB profiles – Wan-shu Wu Bug fixed

Assimilation techniques Hybrid for NAM – Wan-shu Wu Work continues – results using global ensemble encouraging Hybrid for HWRF – Mingjing Tong HWRF implementation using global ensemble on WCOSS Regional Strong Constraint – David Parrish Cloudy Radiance Assimilation – MinJeong Kim, Emily Liu, Yanqiu Zhu, X. Zhang, Will McCarty(GMAO) See previous Add 4d capability to hybrid – Daryl Kleist Included in trunk – Experimentation now showing significant impact Add precip and cloud physics to strong constraint – Min-Jeong Kim, Daryl Kleist Hurricane location ensemble in hybrid – Yoichiro Ota – Russ Treadon

Maintenance and general improvements techniques Maintenance of EMC GSI trunk - Mike Lueken Continues many upgrades completed Improvement of RTMA application – Manuel Pondeca Continues Improvement of NAM analysis – Wan-shu Wu Improvement of GFS analysis – Russ Treadon/Andrew Collard Continues – new resolutions and new observations Radiance monitoring – Ed Safford Included in trunk and operations Unification of data monitoring – Ed Safford Underway FOV updates for new satellite instruments – George Gayno, Ninghai Sun (NESDIS) Code Structure – Mike Lueken, Ricardo Todling (GMAO) Transfer to different machines – Daryl Kleist, Russ Treadon, Others WCOSS transition completed – code much more general

Additional projects beginning for Sandy Supplemental DA Focuses Clouds and precipitation Aircraft data Model parameterizations Increased resolution 4d Hybrid Improved Ensembles www.imsg.com

Final comments

Advantages of community GSI GSI is a well-supported communalized operation data assimilation system Many groups (EMC, GMAO, GSD, MMM,…) are actively working on various new developments for several operation applications (GFS, NMMB, RR, RTMA, …) DTC has built good expertise to support GSI Has a well maintained repository to make the latest code available to friendly users DTC welcome collaboration and support R2O Users play an important role in improving the support

Challenges GSI is a fast evolving system Limited resources to support Hard to follow for some researchers Need to build new expertise for DTC staff Limited resources to support GSI adds many new functions each year Users application areas are extending Reduced funding for support All support staff are shared with other projects Collaboration are important but cost