1. GSI Data Assimilation System
WRF-DA Tutorial, July, 2009, Boulder, CO
GSI Data Assimilation System
Hui Shao, Ming Hu, Laurie Carson,
Louisa Nance, Xiang-Yu Huang, Bill Kuo
Developmental Testbed Center
John Derber, Russ Treadon
NOAA/NCEP/Environmental Modeling Center
Acknowledgement: NOAA, AFWA, NCAR

2. Primary Developers NOAA/NCEP/EMC NASA/GMAO And NOAA/GSD, NCAR/MMM,…
John Derber, Jim Purser, Russ Treadon, Wan-Shu Wu, Dave Parrish, Lidia Cucurull, Dave Parrish, Manuel Pondeca, Paul van Delst, Daryl Kleist, Xiujuan Su, Yanqiu Zhu, and others
NASA/GMAO
Ricardo Todling, Ron Errico, Runhua Yang, Ron Gelaro, Wei Gu, and others
And NOAA/GSD, NCAR/MMM,…

3. Session Outline
GSI System and Community Support (30 minutes, presented by Hui Shao)
Installation, Running, and Diagnostics (30 minutes, presented by Ming Hu)

4. GSI System 4

5. Gridpoint Statistical Interpolation (GSI)
3D variational assimilation (same as WRF-VAR)
J =1/2 (x-xb)TB-1(x-xb) + 1/2(H(x)-y0)T(E+F)-1(H(x)-y0) + JC
Fit to background + Fit to observations + constraints
x = Analysis (Output)
xb = Background (Input)
B = Background error covariance (Input)
H = Forward model (within GSI)
y0 = Observations (Input)
E+F = R = Instrument error + Representativeness error (Input)
JC = Constraint terms (within GSI)

6. Operational GSI Applications
System
Implementation date
Primary Developers
Physical SST retrieval
9/27/2005
NCEP/EMC*
NAM (regional)
6/20/2006
RTMA
8/22/2006
GFS (Global)
5/1/2007
HWRF
6/19/2007
Rapid Refresh (RR)
2010
NOAA/GSD
AFWA operational
2010 or 2011
NCAR/MMM
When RR will be operational?
* Collaborated with NASA/GMAO and others 6

7. Air Force Weather Agency Domains
NAM
HWRF
Rapid Refresh Domain
Air Force Weather Agency Domains

8. Write analysis & related output
GSI Code Flow
Read in & distribute
observations,
background
& background errors
User input &
initializations
Additional
initializations
Outer loop
a) Set up right hand side of analysis equation
(Compute observation innovations)
b) Call inner loop
Compute gradient information
Apply background error
Compute search direction
Compute step size
Update analysis increment
Write analysis & related output

9. Background Fields (xb)
Currently works for the following systems
Global
GFS
GMAO global
Regional
WRF (ARW & NMM) - binary and netcdf
NCEP RTMA

10. Observations(yo) *Some of the data are restricted Radiosondes
Pibal winds
Synthetic tropical cyclone winds
Wind profilers
Conventional aircraft reports
ASDAR aircraft reports
MDCARS aircraft reports**
Dropsondes
MODIS IR and water vapor winds
GMS, METEOSAT and GOES cloud drift IR and visible winds
GOES water vapor cloud top winds
Surface land observations*
Surface ship and buoy observation
SSM/I wind speeds
QuikScat wind speed and direction
SSM/I precipitable water
SSM/I and TRMM TMI precipitation estimates
Doppler radial velocities
VAD (NEXRAD) winds
GPS precipitable water estimates
GPS Radio occultation refractivity profiles
SBUV ozone profiles (other ozone data under test)
*Some of the data are restricted

11. Observations(yo) (cont.)
Regional
GOES-11 and 12 Sounders – thinned to 120km
Channels     Individual fields of view     4 Detectors treated separately     Over ocean only
AMSU-A – thinned to 60km     NOAA-15     Channels 1-10, 12, 15     NOAA-18     Channels 1-8, 10-11, 15
AMSU-B/MHS – thinned to 60km     NOAA-15     Channels 1-3, 5     NOAA-16     Channels 1-5     NOAA-17    Channels 1-5     NOAA-18    Channels 1-5
HIRS – thinned to 120km     NOAA-17    Channels    
Global
GOES-11 and 12 Sounders –thinned to 180km     Channels     Individual fields of view     4 Detectors treated separately     Over ocean only
AMSU-A – thinned to 145km     NOAA-15    Channels 1-10, 12-13, 15     NOAA-18    Channels 1-8, 10-13, 15     METOP       Channels 1-13, 15     AQUA          Channels 1-6, 8-13, 15
AMSU-B/MHS – thinned to 240km     NOAA-15     Channels 1-3, 5     NOAA-16     Channels 1-5     NOAA-17    Channels 1-5     NOAA-18    Channels 1-5     METOP        Channels 1-5
HIRS - thinned to 180km     NOAA-17    Channels     METOP        Channels 2-15
AIRS – thinned to 180km     AQUA        148 Channels

12. Observation Operator (H)
To use observation, GSI simulates observation using analysis variables – observation operator (H)
Can be simple interpolation to ob location/time.
Can be more complex (e.g., radiation transfer).
For radiances, GSI uses CRTM.

13. Background Errors (B)
Space correlation computed using recursive filters in horizontal and vertical
Multivariate relation
Flow dependent variability in background error
Background error variances modified based on 9 and 3 hour forecast differences:
Variance increased in regions of rapid change
Variance decreased in “calm” regions
Global mean variance ~ preserved
Being used for regional (US) surface analysis operationally.

14. Surface pressure background error standard deviation fields
HPC Surface Analysis
b) with flow dependent re-scaling L
rescaled
a) without re-scaling
“as is”
Surface pressure background
error standard deviation
fields
Valid at 00Z November 06, 2007

15. Moisture analysis Option 1: univariate Option 2: multivariate
Temperature (blue) increment forces large increment in RH (shaded).
Option 2: multivariate
Temperature (blue) increment
forces increment in q (red).
much smaller RH (shaded) increment.

16. Observation Errors Improved specification of observational errors
Adaptive Tuning
After tuning
After tuning and smoothing
Before tuning

17. Observation Quality Control
External platform specific QC
Some gross checking in PREPBUFR file creation
Optimal interpolation quality control (OIQC) – on its way out
Analysis QC
Gross checks – specified in input data files
Variational quality control (VarQC) – implemented operationally in Feb 2009
Number of data rejected by OIQC
VarQC weight (W)

18. Bias Correction of Radiance Data
NCEP uses a two step process for Tb
Scan angle correction – based on position
Air mass correction – based on predictors
Predictors
mean
path length (local zenith angle determined)
integrated lapse rate
integrated lapse rate ** 2
cloud liquid water

19. B-O Histogram DMSP15 July2004 : 1month before bias correction
19V
19H
22V
37V
37H
85V
85H
DMSP15 July2004 : 1month
before bias correction
after bias correction

20. Community Support 20

21. Community GSI Code Goals: DTC Research Community
System
Implementation date
Mode
Physical SST retrieval
9/27/2005
CRTM + analytical solution
NAM (regional)
6/20/2006
3D-VAR
RTMA
8/22/2006
2D-VAR
Global
5/1/2007
HWRF
6/19/2007 RR
Early 2010
AFWA operational
2010 or early 2011
DTC
Research Community
When RR will be operational?
Goals:
Provide current operational GSI capability to the research community (O2R)
Provide a framework for distributed development of new capabilities & advances in data assimilation (R2O)

22. Definition of Community Codes
Free and shared resource
Ongoing distributed development by both research and operational communities
Maintained under version control
Periodic releases made available to the community
Includes latest developments of new capabilities and techniques
Centralized support
Provided in collaboration with developers

23. GSI Community Release Timetable
Tasks
Timeline
Note
Beta release (Q1FY09)
Jun, 2009
Friendly user only (through the GSI website)
First release (Q1FY09)
Sep, 2009
Tutorial
Jun, 2010
With user support
Residential tutorial and hand-on practical session

24. Community GSI Code Repository
Proposed Structure
Community Repository
DTC
Developers
Code
Management
Plan
NCEP EMC
Repository
release
Community
Success of this structure will depend on communication and collaboration among all GSI developers, users, and DTC.

25. User’s Website http://www.dtcenter.org/com-GSI/users/
Release announcement
System component descriptions
Documentation
User’s Guide
Presentations
Registration
Software downloads
Bug fix reports
User support information
Tutorial information
Release list? 25

26. GSI User’s Guide
User’s Guide will be updated to be consistent with each new release to the community.

27. On-line Tutorial (Beta Release)

28. On-line Documents (Beta Release)

29. GSI Web Brower

30. O2R2O2R2O2R2O2R2O …
Which one to choose? clip

31. GSI: Compile 31
2008 Joint WRF Tutorial

32. Requirements System required libraries GSI system
FORTRAN 90/95 compiler
C compiler
Perl
netCDF
GSI system
Download GSI system tar files (GSIbeta.tar.gz ) from
gunzip and untar
tar –zxvf GSIbeta.tar.gz (Should see GSI/. directory )
cd to GSI directory
cd GSI 32
2008 Joint WRF Tutorial

33. GSI Directory Compile rules Compile scripts Source code directory Run
sorc
makefile
mains
libs
bacio
crtm_gfsgsi
bufr
sfcio
gfsio
sigio w3
mpeu sp
2008 Joint WRF Tutorial

34. Set environment setenv WRF_DIR /home/user/WRFV3 WRF_DIR
WRF needs to be compiled prior to compiling GSI
GSI uses WRF I/O API libraries to do file input and output
WRF directory specified:
setenv WRF_DIR /home/user/WRFV3
netCDF
If netCDF libraries are not located in the standard /usr/local , then
setenv NETCDF “path for netCDF”
For LINUX systems, make sure the netCDF libraries are installed using the same compiler (PGI, Intel, g95) that will be used to compile GSI. 34
2008 Joint WRF Tutorial

35. Configuring GSI To create a GSI configuration file for your computer:
./configure
This script checks the system hardware and software (mostly netCDF), and then offers the user choices for configuring GSI:
Choices for 32-bit LINUX operated machines are:
1. Linux i486 i586 i686, PGI compiler
2. Linux i486 i586 i686, Intel compiler
3. Linux i486 i586 i686, gfortran compiler
Choices for IBM machines are:
1. AIX xlf compiler with xlc 35
2008 Joint WRF Tutorial

36. Configuring GSI, cont. configure.gsi
Created by the ./configure command
contains compilation options, rules, etc.
specific to your computer
can be edited to change compile options, if desired.
At this time, the IBM option is well tested. Working on Linux option test.
The arch/configure.defaults file can be edited to add a new option if needed. 36
2008 Joint WRF Tutorial

37. Compiling GSI To compile: To get compile help message:
./compile >& compile_gsi.log
To get compile help message:
./compile -h
If the compilation is successful, it will create one executable under bin/:
gsi.exe 37
2008 Joint WRF Tutorial

38. To remove all object files and executables
Clean Compilation
To remove all object files and executables
./clean
To remove all built files, including configure.gsi
./clean –a
Clean is recommended if
compilation failed
want to change configuration file 38
2008 Joint WRF Tutorial

39. Running GSI User’s Guide: Chapter 339

40. To run GSI, you need: GSI Executable Background (first guess) file
Observations
Not needed for single observation experiment
Fixed files (within GSI package)
Run script (namelist included) 40

41. Background
GSI can use
WRF NMM input file in binary format
WRF NMM input file in netcdf format
WRF ARW input file in binary format
WRF ARW input file in netcdf format
GFS input file in binary format
GMAO global model input file in binary format
DTC has only tested regional analysis with WRF input
On IBM: both binary and nedcdf format
On Linux: netcdf format only 41

42. Observation All observations have to be in BUFR format
prepbufr: NCEP flavor BUFR
Need NCEP BUFR library 42

43. Fixed files
Collection of statistic and control files under fix directory
Background and observation errors
berror_stats, errtable
Observation data control file (info files)
convinfo , satinfo
Bias correction used by radiance analysis
satbias_angle, satbias_in
Radiance coefficient used by CRTM
EmisCoeff.bin, CloudCoeff.bin 43

44. Run script - structure Ask for computer resources to run GSI
Set environment variables for the machine
Set experiments variables (experiment name, analysis time, background and observation)
Check the definition of required variables
Generate a run directory for GSI (working or temp directory)
Copy GSI executable to run directory
Copy background file to run directory
Copy or link observations to run directory
Copy fixed files to run directory
Generate namelist for GSI
Run the GSI executable
Save the GSI analysis results 44

45. Run Script - Experiment variables
# analysis time (YYYYMMDDHH)
ANAL_TIME=
# working direcotry, where GSI runs
WORK_ROOT=./gsi/case
# path and name of background file
BK_FILE=./ /bkARW/wrfout_d01_ _12:00:00
# path of observations
OBS_ROOT=./gsi/case
PREPBUFR=./tutorialcases/data/newgblav.gdas1.t12z.prepbufr.nr
# path of fix files
FIX_ROOT=./GSI/fix
# path and name of the gsi executable
GSI_EXE=./GSI/bin/gsi.exe
# which background error covariance and parameter will be used (GLOBAL or NAM)
bkcv_option=NAM 45

46. Run Script - run directory
Create a new directory for each run
Copy or link data
Executable
Background
Must be copied, will be over-written by analysis result
Observations
Link or touch if not exist
Fixed files
Link 46

47. Run Script : namelist Generated by running scripts
Change namelist by editing run script
Works for both global and regional analysis
Detailed explanation in section 3.4
&SETUP
miter=2,niter(1)=5,niter(2)=5,
write_diag(1)=.true.,write_diag(2)=.false.,write_diag(3)=.true.,
qoption=1, gencode=78,factqmin=0.005,factqmax=0.005,deltim=$DELTIM,
ndat=59,npred=5,iguess=-1, oneobtest=.false.,retrieval=.false.,l_foto=.false.,
use_pbl=.false.,
$SETUP /
&GRIDOPTS
JCAP=$JCAP,NLAT=$NLAT,NLON=$LONA,nsig=$LEVS,hybrid=.true.,
wrf_nmm_regional=.false.,wrf_mass_regional=.true., diagnostic_reg=.false.,
filled_grid=.false.,half_grid=.true.,netcdf=.true.,
regional=.true.,nlayers(63)=3,nlayers(64)=6,
$GRIDOPTS 47

48. Tuning and Diagnostics User’s Guide: Chapter 448 48

49. Standard out file - stdout
Includes lots of important information
First place to look after any GSI run
If successful
Data distribution
Optimal iteration
Maximum and minimum of analysis fields
If fails,
which part of GSI has problem
what is the possible reason for failure 49 49

50. stdout - structure read in all data and prepare analysis:
read in configuration (namelist)
read in background
read in constant file (fixed file)
read in observation
partition background and observation data for parallel analysis
optimal iteration (analysis)
save analysis result 50

51. stdout – example(1) Top of stdout: read in background End of stdout:
0: rmse_var=T
0: ordering=XYZ
0: WrfType,WRF_REAL=
0: ndim1= 3
0: staggering= N/A
0: start_index=
0: end_index=
0: k,max,min,mid T=
0: k,max,min,mid T=
……….
0: k,max,min,mid T=
Top of stdout:
read in background
End of stdout:
write out analysis results
0: Read_SpcCoeff_Binary(INFORMATION) : FILE: amsua_n15.SpcCoeff.bin;
0: SpcCoeff RELEASE.VERSION: N_CHANNELS=15
0: Read_SpcCoeff_Binary(INFORMATION) : FILE: amsub_n15.SpcCoeff.bin;
0: SpcCoeff RELEASE.VERSION: N_CHANNELS=5
0: Read_SpcCoeff_Binary(INFORMATION) : FILE: hirs3_n16.SpcCoeff.bin;
0: SpcCoeff RELEASE.VERSION: N_CHANNELS=19
0: Read_SpcCoeff_Binary(INFORMATION) : FILE: amsua_n16.SpcCoeff.bin;
Read CRTM
coefficient 51

52. stdout – example (2) 0: PROGRAM GSI_ANL HAS ENDED. IBM RS/6000 SP
6: READ_LIDAR: bufr file date is : READ_AIRS: bufr file date is : READ_RADAR: vad wind bufr file date is : READ_PREPBUFR: bufr file date is : READ_BUFRTOVS: bufr file date is mhsbufr 5: n,lat,lon,qm= : READ_PREPBUFR: messages/reports = 872 / ntread = 1
Read observations
0: 0: GLBSOI: START pcgsoi
0: penalty,grad ,a,b= E E E E+00
0: pnorm,gnorm, step? E E+01 good
0: penalty,grad ,a,b= E E E E+00
0: pnorm,gnorm, step? E E+00 good
Iteration
0: PROGRAM GSI_ANL HAS ENDED. IBM RS/6000 SP 52

53. Single Observation Test
A good way to check
ratio of background and observation variance
pattern of background error covariance
Setup in namelist
SETUP
oneobtest=.true.,
SINGLEOB_TEST
maginnov=1.0,
magoberr=1.0,
oneob_type='t',
oblat=20.,
oblon=285.,
obpres=850.,
obdattim= ,
obhourset=0., 53 53

54. Control Data Usage From namelist and the links of observation files
dfile(01)='prepbufr‘, dtype(01)='ps‘, dplat(01)=' ', dsis(01)='ps'
dfile(02)='prepbufr‘ dtype(02)='t', dplat(02)=' ', dsis(02)='t',
dfile(27)='amsuabufr‘,dtype(27)='amsua‘,dplat(27)='n15‘,dsis(27)='amsua_n15',
dfile(28)='amsuabufr‘,dtype(28)='amsua‘,dplat(28)='n16‘,dsis(28)='amsua_n16',
From info file (convinfo …)
otype type sub iuse twindow numgrp ngroup nmiter gross ermax ermin var_b var_pg ps ps ps ps ps ps ps ps t t t t t 54 54

55. Observation fit - statistic files55

56. Convergence Information
stdout
0: 0: GLBSOI: START pcgsoi
0: penalty,grad ,a,b= E E E E+00
0: pnorm,gnorm, step? E E+01 good
0: penalty,grad ,a,b= E E E E+00
0: pnorm,gnorm, step? E E+00 good
More detail in fort.220
J= E E E+00
E E E+04
E E E+04
E E E+00
E E+04
20 items in cost function:
5 wind observation satellite radiance observation temperature observation
8 precipitable water obs specific humidity obs surface pressure observation 56 56

57. Questions, Comments, Suggestions57