1. 在科研中更好的使用美国业务资料分析系统（GSI）
Ming Hu   *Developmental Testbed Center, NCAR-NOAA/GSD **Cooperative Institute for Research in Environmental Sciences, Colorado University at Boulder  

2. Outline
General introduction to the Gridpoint Statistical Interpolation (GSI) system
Development Testbed Center and Community GSI
History, repository, test, and review committee
Available GSI resources for community
O2R: website, document, tutorial, and help desk
R2O: community contributions
Current and future development
Community GSI and operational GSI

3. General introduction to the Gridpoint Statistical Interpolation system
This section is mainly based on John Derber’s talk
in 2013 summer GSI tutorial

4. John C. Derber NOAA/NWS/NCEP/EMC
Overview of GSI
John C. Derber
NOAA/NWS/NCEP/EMC

5. History
The Spectral Statistical Interpolation (SSI) analysis system was developed at NCEP in the late 1980’s and early 1990’s.
Main advantages of this system over OI systems were:
All observations are used at once (much of the noise generated in OI analyses was generated by data selection)
Ability to use forward models to transform from analysis variable to observations
Analysis variables can be defined to simplify covariance matrix and are not tied to model variables (except need to be able to transform to model variable)
The SSI system was the first operational
variational analysis system
system to directly use radiances

6. History
While the SSI system was a great improvement over the prior OI system – it still had some basic short- comings
Since background error was defined in spectral space – not simple to use for regional systems
Diagonal spectral background error did not allow much spatial variation in the background error
Not particularly well written since developed as a prototype code and then implemented operationally

7. History
The Global Statistical Interpolation (GSI) analysis system was developed as the next generation global/regional analysis system
Wan-Shu Wu, R. James Purser, David Parrish
Three-Dimensional Variational Analysis with spatially Inhomogeneous Covariances. Mon. Wea. Rev., 130,
Based on SSI analysis system
Replace spectral definition for background errors with grid point version based on recursive filters

8. History Used in NCEP operations for Operational at AFWA
Regional
Global
Hurricane
Real-Time Mesoscale Analysis
Rapid Refresh (ESRL/GSD)
Operational at AFWA
GMAO collaboration
Modification to fit into WRF and NCEP infrastructure
Evolution to ESMF

9. General Comments GSI analysis code is an evolving system.
Scientific advances
situation dependent background errors -- hybrid
new satellite data
new analysis variables
Improved coding
Bug fixes
Removal of unnecessary computations, arrays, etc.
More efficient algorithms (MPI, OpenMP)
Bundle structure
Generalizations of code
Different compute platforms
Different analysis variables
Different models
Improved documentation
Fast evolution creates difficulties for slower evolving research projects

10. General Comments Code is intended to be used Operationally
Must satisfy coding requirements
Must fit into infrastructure
Must be kept as simple as possible
External usage intended to:
Improve external testing
Reduce transition to operations work/time
Reduce duplication of effort

11. Simplification to operational 3-D for presentation
For today’s introduction, I will be talking about using the GSI for standard operational 3-D var. analysis. Many other options available or under development
4d-var
hybrid assimilation
observation sensitivity
FOTO
Additional observation types
SST retrieval
Detailed options
Options make code more complex – difficult balance between options and simplicity

12. Basic analysis problem
J = Jb + Jo + Jc
J = (x-xb)TB-1(x-xb) + (H(x)-y0)T(E+F)-1(H(x)-y0) + JC
J = Fit to background + Fit to observations + constraints
x = Analysis
xb = Background
B = Background error covariance
H = Forward model
y0 = Observations
E+F = R = Instrument error + Representativeness error
JC = Constraint terms

13. Analysis variables
Background errors must be defined in terms of analysis variable
Streamfunction (Ψ)
Unbalanced Velocity Potential (χunbalanced)
Unbalanced Temperature (Tunbalanced)
Unbalanced Surface Pressure (Psunbalanced)
Ozone – Clouds – etc.
Satellite bias correction coefficients

14. Analysis variables χ = χunbalanced + A Ψ T = Tunbalanced + B Ψ
Ps = Psunbalanced + C Ψ
Streamfunction is a key variable defining a large percentage T and Ps (especially away from equator). Contribution to χ is small except near the surface and tropopause.

15. Background fields Current works for following systems
NCEP GFS
NCEP NMM – binary and netcdf
NCEP RTMA
NCEP Hurricane (not using subversion version yet)
GMAO global
ARW – binary and netcdf – (not operationally used yet RR - GSD)
FGAT (First Guess at Appropriate Time) enabled up to 100 time levels

16. Background Errors Three paths – more in talk by D. Kleist
Isotropic/homogeneous
Most common usage.
Function of latitude/height
Vertical and horizontal scales separable
Variances can be location dependent
Anisotropic/inhomogeneous
Function of location /state
Can be full 3-D covariances
Still relatively immature
Hybrid

17. Observations
Observational data is expected to be in BUFR format (this is the international standard)
Each observation type (e.g., u,v,radiance from NOAA-15 AMSU-A) is read in on a particular processor or group of processors (parallel read)
Data thinning can occur in the reading step.
Checks to see if data is in specified data time window and within analysis domain

18. Input data – Satellite currently used
Regional
AMSU-A
NOAA-15 Channels 1-10, 12-13, 15
NOAA-18 Channels 1-8, 10-13, 15
METOP-A Channels1-6, 8-13, 15
AQUA Channels 5, 8-13
Thinned to 60km
AMSU-B/MHS
NOAA-15 Channels 1-3, 5
NOAA-18 Channels 1-5
METOP-A Channels 1-5
HIRS
NOAA-17 Channels 2-15
METOP-A Channels 2-15
Thinned to 120km
AIRS
AQUA 148 Channels
 IASI
METOP-A 165 Channels
Global
all thinned to 145km
GOES-15 Sounders
Channels 1-15
Individual fields of view
4 Detectors treated separately
Over ocean only
AMSU-A
NOAA-15 Channels 1-10, 12-13, 15
NOAA-18 Channels 1-8, 10-13, 15
NOAA Channels 1-7, 9-13, 15
METOP-A Channels 1-6, 8-13, 15
AQUA Channels 6, 8-13
ATMS
NPP Channels 1-14,16-22
MHS
NOAA-18 Channels 1-5
NOAA-19 Channels 1-5
METOP-A Channels 1-5
HIRS
METOP-A Channels 2-15
AIRS
AQUA 148 Channels
IASI
METOP-A Channels

19. Input data – Conventional currently used
Radiosondes
Pibal winds
Synthetic tropical cyclone winds
wind profilers
conventional aircraft reports
ASDAR aircraft reports
MDCARS aircraft reports
dropsondes
MODIS IR and water vapor winds
GMS, JMA, METEOSAT and GOES cloud drift IR and visible winds
GOES water vapor cloud top winds
Surface land observations
Surface ship and buoy observation
SSM/I wind speeds
QuikScat and ASCATwind speed and direction
SSM/I and TRMM TMI precipitation estimates
Doppler radial velocities
VAD (NEXRAD) winds
GPS precipitable water estimates
GPS Radio occultation refractivity and bending angle profiles
SBUV ozone profiles and OMI total ozone

20. Simulation of observations
To use observation, must be able to simulate observation
Can be simple interpolation to ob location/time
Can be more complex (e.g., radiative transfer)
For radiances we use CRTM
Vertical resolution and model top important

21. Quality control External platform specific QC
Some gross checking in PREPBUFR file creation
Analysis QC
Gross checks – specified in input data files
Variational quality control
Data usage specification (info files)
Outer iteration structure allows data rejected (or downweighted) initially to come back in
Ob error can be modified due to external QC marks
Radiance QC much more complicated. Tomorrow!

22. Useful References
Wan-Shu Wu, R. James Purser and David F. Parrish, 2002: Three-Dimensional Variational Analysis with Spatially Inhomogeneous Covariances. Monthly Weather Review, Vol. 130, No. 12, pp. 2905–2916. 
R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part I: Spatially Homogeneous and Isotropic Gaussian Covariances. Monthly Weather Review, Vol. 131, No. 8, pp. 1524–1535. 
R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part II: Spatially Inhomogeneous and Anisotropic General Covariances. Monthly Weather Review, Vol. 131, No. 8, pp. 1536–1548. 
McNally, A.P., J.C. Derber, W.-S. Wu and B.B. Katz, 2000: The use of TOVS level-1B radiances in the NCEP SSI analysis system.  Q.J.R.M.S., 126,
Parrish, D. F. and J. C. Derber, 1992: The National Meteorological Center's spectral statistical interpolation analysis system. Mon. Wea. Rev., 120,
Derber, J. C. and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126,
Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Wu, Wan-Shu; Lord, Stephen , Introduction of the GSI into the NCEP Global Data Assimilation System, Weather and Forecasting. Vol. 24, no. 6, pp Dec 2009
Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Errico, Ronald M; Yang, Runhua, Improving Incremental Balance in the GSI 3DVAR Analysis System, Monthly Weather Review [Mon. Weather Rev.]. Vol. 137, no. 3, pp Mar 2009.
Kazumori, M; Liu, Q; Treadon, R; Derber, JC, Impact Study of AMSR-E Radiances in the NCEP Global Data Assimilation System Monthly Weather Review,Vol. 136, no. 2, pp Feb 2008.
Zhu, Y; Gelaro, R, Observation Sensitivity Calculations Using the Adjoint of the Gridpoint Statistical Interpolation (GSI) Analysis System, Monthly Weather Review. Vol. 136, no. 1, pp Jan 2008.
DTC GSI documentation (

23. DTC comments GSI is an operation system PrepBUFR and NCEP QC procedure
Well-tested for operation applications
Very strict data QC
Advanced operator for radiance
Bias correction is very important radiance data analysis
Model top and channel selection
Data coverage for regional application
PrepBUFR and NCEP QC procedure
The operation system supported by DTC for community users

24. DTC and Community GSI Who is DTC Community GSI: Goal and Building
Community GSI repository
Test and evaluation
GSI review committee

25. Who is DTC ? Developmental Testbed Center (DTC)
Bridge between research and operational community
NWP systems supported by DTC
WRF, HWRF, GSI, MET, …

26. Community GSI: Goals
Provide current operation GSI capability to research community (O2R)
Provide a pathway for research community to contribute operation GSI (R2O)
Provide a framework to enhance the collaboration from distributed GSI developers
Provide rational basis to operational centers and research community for enhancement of data assimilation systems

27. Community GSI: Building
O2R:
Code portability: well-tested GSI code package for release
Documentation: GSI User’s Guide
GSI User’s Webpage
Annual release
On-site Tutorial (lectures and Practical cases)
Help desk
R2O:
Repository: Build, maintain, sync, test
Commit community contributions
GSI Review Committee
The important role of GSI repository and review committee

28. GSI Code Repository Branch GSI Trunk Tag Branch NDAS GDAS
Community release
HWRF
Use tags or branches for: Release, new development, bug fix …
Applications may use different revisions in the trunk (“snapshot”).
Which GSI should I use ?
There is no “DTC GSI”, “EMC GSI” or “global GSI”. There is only one GSI! For a researcher, community release should be sufficient to use. If you are interested in getting new development back to the GSI trunk, contact GSI helpdesk get access to the developmental version of GSI.

29. GSI Review Committee Branch GSI Trunk NCEP/EMC NASA/GMAO NOAA/ESRL
NESDIS
DTC
AFWA
NCAR/MMM
Branch
Tag
GSI Review Committee
Represent research and operational community
Coordinate distributed GSI development
Conduct GSI Code Review
DTC is working as the pathway between the research community and operational community.
Provide visitor program for potential contributions to the operational code.
Provide equivalent-operational tests of community contributions and provide rational basis for operational implementations.

30. History First Community GSI tutorial Created GSI Review Committee
June, 2006:North American Mesoscale (NAM) System, NCEP
May, 2007: Global Forecast System (GFS), NCEP
2007: DTC started to document GSI.
2009:
Created GSI code repository over NCEP and DTC
First GSI release V1.0
Started user support
2010:
First Community GSI tutorial
Created GSI Review Committee
2011
First Community GSI workshop

32. Available GSI resources for community: O2R
User’s Webpage
Documentation
Annual release package
Annual residential tutorial
Help desk

33. Community GSI – User’s Page
Support mainly through User’s Page and help desk:

34. Community GSI Release and Tutorial
The GSI User’s Guide and the on-line tutorial match with official release code
Summer 2013 GSI Tutorial:
NCEP, Aug. 5-7, 2013

35. Community GSI Release
Source code and fixed files were based on:
the GSI EMC trunk r19180 (May 10, 2012)
the community GSI trunk r826
The GSI User’s Guide and the on-line tutorial cases are updated with each official release.

36. Community GSI - Documents
User’s Guide
Matching with each
official release
Tutorial lectures
Code browser
Calling tree
Publications

37. Community GSI- User’s Guide V3.2

38. Community GSI- User’s Guide V3.2

39. Community GSI - Practice
On-line tutorial for each release
Residential tutorial practice cases

40. Community GSI – tutorial and workshop

41. Community GSI - Help desk
Any GSI related questions
Most of questions were answered by DTC staff
Forward complex questions to GSI developers

42. Available GSI resources for community: R2O
Commit the community contributions to trunk
Help desk

43. GSI R2O Transition Procedure
(2011 Implementation)
GSI Review Committee Scientific Review
Development, testing and merging
GSI Review Committee code and commitment review
Community research
Code development candidate 1
Code commitment candidate
(Branches) 2 3
Code in repository trunk

44. GSI R2O applications
DTC contributes to R2O transition, coordinate & assist code commitment:
Mariusz Pagowski: Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model
DTC: Portability issues from trunk head testing
Mariusz Pagowski: Add WRF-Chem model file as background for PM2.5 assimilation
Tom Augline: Add control and state variables for Cloudy analysis (triggered more work on the bundles of GMAO)
Zhiquan Liu: aerosol work has been merged to the trunk
GSD: Recent GSI developments for RR

45. Collaboration HFIP program: BUFR processing training
AMB branch: GSI application for Rapid Refresh
Mariusz Pagowski (AMB): Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model
Don Birkenheuer (FAB): Thorpex work using GSI
EMC GSI group (feedback from community GSI users)
NCAR/MMM DA group (hybrid and cloud analysis,, chemistry analysis )
Space Weather Prediction Center /NOAA (Houjun Wang ) (whole atmosphere data assimilation using GSI)
Other GSI users (OU regional ENKF, PSD (Jeff Whitaker) global ENKF …)

46. R2O Summary
DTC also works with researchers to bring research community contributions back to the GSI operation repository (R2O)
Create Trac to keep community researchers in developing loop
Help researchers to review and merge the code on top of the trunk
Test the code with regression test suite and initial the code review for committing
Send your questions to

47. Current and future development
Community GSI
Operational GSI
August 9th, 2013
Committee Meeting Agenda
8:45 GMAO 
9:30 NESDIS 
10:15 break 
10:25 ESRL 
11:10 NCAR 
11:55 lunch
12:40 DTC 
1:10 NCEP 
3:00 Adjourn

48. Community GSI Maintaining the current support
GSI review committee
Community GSI trunk: sync, test
Annual release with code and updated document
Improving GSI user’s Website
Holding on-site tutorial
Maintaining Helpdesk
Help committing community contributions
Application tools and new instructions to GSI (bundle, hybrid, …)
Build operation-similar test system (ongoing)

49. Ongoing and future EMC GSI development work
EMC GSI development team
Update Aug. 8, 2013

50. Use of observations
Prepare for METOP-B and CrIS implementation - Andrew Collard
Implementation Aug. 20 to WCOSS
Includes SEVIRI from Meteosat-10
Use of AURA MLS ozone profiles – Haixia Liu
Testing of new version of real time data underway
Cloudy Radiance Assimilation – Emily Liu, Yanqiu Zhu, Z. Zhang, Will McCarty (GMAO)
Updates to trunk – Significant progress – Testing without convection
Improved structure of radiance diagnostic files – Andrew Collard - TBD
Satellite wind usage upgrade –XiuJuan Su
GOES hourly wind project
Ground based Doppler radar data – Shun Liu
Conversion to dual-pol
Aircraft based Doppler radar data – Mingjing Tong
In operations on WCOSS
SSMI/S data – Andrew Collard, Emily Liu, and Ellen Liang (NESDIS)
To be included in Spring 2014 implementation
Mesoscale surface observation – Manuel Pondeca, Steve Levine
GLERL surface reduction for lakes under testing
GOES-R/SEVERI radiances – Haixia Liu
Additional quality control under development
Channels used above low clouds
CRTM – Paul VanDelst, David Groff, Quanhua Liu(NESDIS), Yong Chen(NESDIS)
Improvements continue
Improve use of significant levels in RAOB profiles – Wan-shu Wu
Bug fixed

51. Assimilation techniques
Hybrid for NAM – Wan-shu Wu
Work continues – results using global ensemble encouraging
Hybrid for HWRF – Mingjing Tong
HWRF implementation using global ensemble on WCOSS
Regional Strong Constraint – David Parrish
Cloudy Radiance Assimilation – MinJeong Kim, Emily Liu, Yanqiu Zhu, X. Zhang, Will McCarty(GMAO)
See previous
Add 4d capability to hybrid – Daryl Kleist
Included in trunk – Experimentation now showing significant impact
Add precip and cloud physics to strong constraint – Min-Jeong Kim, Daryl Kleist
Hurricane location ensemble in hybrid – Yoichiro Ota – Russ Treadon

52. Maintenance and general improvements techniques
Maintenance of EMC GSI trunk - Mike Lueken
Continues many upgrades completed
Improvement of RTMA application – Manuel Pondeca
Continues
Improvement of NAM analysis – Wan-shu Wu
Improvement of GFS analysis – Russ Treadon/Andrew Collard
Continues – new resolutions and new observations
Radiance monitoring – Ed Safford
Included in trunk and operations
Unification of data monitoring – Ed Safford
Underway
FOV updates for new satellite instruments – George Gayno, Ninghai Sun (NESDIS)
Code Structure – Mike Lueken, Ricardo Todling (GMAO)
Transfer to different machines – Daryl Kleist, Russ Treadon, Others
WCOSS transition completed – code much more general

53. Additional projects beginning for Sandy Supplemental
DA Focuses
Clouds and precipitation
Aircraft data
Model parameterizations
Increased resolution
4d Hybrid
Improved Ensembles

54. Final comments

55. Advantages of community GSI
GSI is a well-supported communalized operation data assimilation system
Many groups (EMC, GMAO, GSD, MMM,…) are actively working on various new developments for several operation applications (GFS, NMMB, RR, RTMA, …)
DTC has built good expertise to support GSI
Has a well maintained repository to make the latest code available to friendly users
DTC welcome collaboration and support R2O
Users play an important role in improving the support

56. Challenges GSI is a fast evolving system Limited resources to support
Hard to follow for some researchers
Need to build new expertise for DTC staff
Limited resources to support
GSI adds many new functions each year
Users application areas are extending
Reduced funding for support
All support staff are shared with other projects
Collaboration are important but cost