Presentation is loading. Please wait.

Presentation is loading. Please wait.

Ming Hu * Developmental Testbed Center, NCAR-NOAA/GSD **Cooperative Institute for Research in Environmental Sciences, Colorado University at Boulder 在科研中更好的使用美国业务资料分析系统.

Similar presentations

Presentation on theme: "Ming Hu * Developmental Testbed Center, NCAR-NOAA/GSD **Cooperative Institute for Research in Environmental Sciences, Colorado University at Boulder 在科研中更好的使用美国业务资料分析系统."— Presentation transcript:

1 Ming Hu * Developmental Testbed Center, NCAR-NOAA/GSD **Cooperative Institute for Research in Environmental Sciences, Colorado University at Boulder 在科研中更好的使用美国业务资料分析系统 ( GSI )

2 Outline General introduction to the Gridpoint Statistical Interpolation (GSI) system Development Testbed Center and Community GSI History, repository, test, and review committee Available GSI resources for community O2R: website, document, tutorial, and help desk R2O: community contributions Current and future development Community GSI and operational GSI 2

3 General introduction to the Gridpoint Statistical Interpolation system This section is mainly based on John Derber’s talk in 2013 summer GSI tutorial 3

4 Overview of GSI John C. Derber NOAA/NWS/NCEP/EMC 4

5 History The Spectral Statistical Interpolation (SSI) analysis system was developed at NCEP in the late 1980’s and early 1990’s. Main advantages of this system over OI systems were: All observations are used at once (much of the noise generated in OI analyses was generated by data selection) Ability to use forward models to transform from analysis variable to observations Analysis variables can be defined to simplify covariance matrix and are not tied to model variables (except need to be able to transform to model variable) The SSI system was the first operational variational analysis system system to directly use radiances 5

6 History While the SSI system was a great improvement over the prior OI system – it still had some basic short- comings Since background error was defined in spectral space – not simple to use for regional systems Diagonal spectral background error did not allow much spatial variation in the background error Not particularly well written since developed as a prototype code and then implemented operationally 6

7 History The Global Statistical Interpolation (GSI) analysis system was developed as the next generation global/regional analysis system Wan-Shu Wu, R. James Purser, David Parrish Three-Dimensional Variational Analysis with spatially Inhomogeneous Covariances. Mon. Wea. Rev., 130, Based on SSI analysis system Replace spectral definition for background errors with grid point version based on recursive filters 7

8 History Used in NCEP operations for Regional Global Hurricane Real-Time Mesoscale Analysis Rapid Refresh (ESRL/GSD) Operational at AFWA GMAO collaboration Modification to fit into WRF and NCEP infrastructure Evolution to ESMF 8

9 General Comments GSI analysis code is an evolving system. Scientific advances situation dependent background errors -- hybrid new satellite data new analysis variables Improved coding Bug fixes Removal of unnecessary computations, arrays, etc. More efficient algorithms (MPI, OpenMP) Bundle structure Generalizations of code Different compute platforms Different analysis variables Different models Improved documentation Fast evolution creates difficulties for slower evolving research projects 9

10 General Comments Code is intended to be used Operationally Must satisfy coding requirements Must fit into infrastructure Must be kept as simple as possible External usage intended to: Improve external testing Reduce transition to operations work/time Reduce duplication of effort 10

11 Simplification to operational 3-D for presentation For today’s introduction, I will be talking about using the GSI for standard operational 3-D var. analysis. Many other options available or under development 4d-var hybrid assimilation observation sensitivity FOTO Additional observation types SST retrieval Detailed options Options make code more complex – difficult balance between options and simplicity 11

12 Basic analysis problem J = J b + J o + J c J = (x-x b ) T B -1 (x-x b ) + (H(x)-y 0 ) T (E+F) -1 (H(x)-y 0 ) + J C J = Fit to background + Fit to observations + constraints x= Analysis x b = Background B= Background error covariance H= Forward model y 0 = Observations E+F= R = Instrument error + Representativeness error J C = Constraint terms 12

13 Analysis variables Background errors must be defined in terms of analysis variable Streamfunction (Ψ) Unbalanced Velocity Potential (χ unbalanced ) Unbalanced Temperature (T unbalanced ) Unbalanced Surface Pressure (Ps unbalanced ) Ozone – Clouds – etc. Satellite bias correction coefficients 13

14 Analysis variables χ = χ unbalanced + A Ψ T = T unbalanced + B Ψ Ps = Ps unbalanced + C Ψ Streamfunction is a key variable defining a large percentage T and P s (especially away from equator). Contribution to χ is small except near the surface and tropopause. 14

15 Background fields Current works for following systems NCEP GFS NCEP NMM – binary and netcdf NCEP RTMA NCEP Hurricane (not using subversion version yet) GMAO global ARW – binary and netcdf – (not operationally used yet RR - GSD) FGAT (First Guess at Appropriate Time) enabled up to 100 time levels 15

16 Background Errors Three paths – more in talk by D. Kleist Isotropic/homogeneous Most common usage. Function of latitude/height Vertical and horizontal scales separable Variances can be location dependent Anisotropic/inhomogeneous Function of location /state Can be full 3-D covariances Still relatively immature Hybrid 16

17 Observations Observational data is expected to be in BUFR format (this is the international standard) Each observation type (e.g., u,v,radiance from NOAA-15 AMSU-A) is read in on a particular processor or group of processors (parallel read) Data thinning can occur in the reading step. Checks to see if data is in specified data time window and within analysis domain 17

18 Input data – Satellite currently used Regional AMSU-A NOAA-15 Channels 1-10, 12-13, 15 NOAA-18 Channels 1-8, 10-13, 15 METOP-AChannels1-6, 8-13, 15 AQUAChannels 5, 8-13 Thinned to 60km AMSU-B/MHS NOAA-15 Channels 1-3, 5 NOAA-18Channels 1-5 METOP-AChannels 1-5 Thinned to 60km HIRS NOAA-17Channels 2-15 METOP-AChannels 2-15 Thinned to 120km AIRS AQUA148 Channels Thinned to 120km IASI METOP-A165 Channels Global all thinned to 145km GOES-15 Sounders Channels 1-15 Individual fields of view 4 Detectors treated separately Over ocean only AMSU-A NOAA-15 Channels 1-10, 12-13, 15 NOAA-18 Channels 1-8, 10-13, 15 NOAA-19 Channels 1-7, 9-13, 15 METOP-A Channels 1-6, 8-13, 15 AQUA Channels 6, 8-13 ATMS NPPChannels 1-14,16-22 MHS NOAA-18 Channels 1-5 NOAA-19Channels 1-5 METOP-AChannels 1-5 HIRS METOP-AChannels 2-15 AIRS AQUA148 Channels IASI METOP-A 165 Channels 18

19 Input data – Conventional currently used Radiosondes Pibal winds Synthetic tropical cyclone winds wind profilers conventional aircraft reports ASDAR aircraft reports MDCARS aircraft reports dropsondes MODIS IR and water vapor winds GMS, JMA, METEOSAT and GOES cloud drift IR and visible winds GOES water vapor cloud top winds Surface land observations Surface ship and buoy observation SSM/I wind speeds QuikScat and ASCATwind speed and direction SSM/I and TRMM TMI precipitation estimates Doppler radial velocities VAD (NEXRAD) winds GPS precipitable water estimates GPS Radio occultation refractivity and bending angle profiles SBUV ozone profiles and OMI total ozone 19

20 Simulation of observations To use observation, must be able to simulate observation Can be simple interpolation to ob location/time Can be more complex (e.g., radiative transfer) For radiances we use CRTM Vertical resolution and model top important 20

21 Quality control External platform specific QC Some gross checking in PREPBUFR file creation Analysis QC Gross checks – specified in input data files Variational quality control Data usage specification (info files) Outer iteration structure allows data rejected (or downweighted) initially to come back in Ob error can be modified due to external QC marks Radiance QC much more complicated. Tomorrow! 21

22 Useful References Wan-Shu Wu, R. James Purser and David F. Parrish, 2002: Three-Dimensional Variational Analysis with Spatially Inhomogeneous Covariances. Monthly Weather Review, Vol. 130, No. 12, pp. 2905–2916. R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part I: Spatially Homogeneous and Isotropic Gaussian Covariances. Monthly Weather Review, Vol. 131, No. 8, pp. 1524–1535. R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part II: Spatially Inhomogeneous and Anisotropic General Covariances. Monthly Weather Review, Vol. 131, No. 8, pp. 1536–1548. McNally, A.P., J.C. Derber, W.-S. Wu and B.B. Katz, 2000: The use of TOVS level-1B radiances in the NCEP SSI analysis system. Q.J.R.M.S., 126, Parrish, D. F. and J. C. Derber, 1992: The National Meteorological Center's spectral statistical interpolation analysis system. Mon. Wea. Rev., 120, Derber, J. C. and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Wu, Wan-Shu; Lord, Stephen, Introduction of the GSI into the NCEP Global Data Assimilation System, Weather and Forecasting. Vol. 24, no. 6, pp Dec 2009 Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Errico, Ronald M; Yang, Runhua, Improving Incremental Balance in the GSI 3DVAR Analysis System, Monthly Weather Review [Mon. Weather Rev.]. Vol. 137, no. 3, pp Mar Kazumori, M; Liu, Q; Treadon, R; Derber, JC, Impact Study of AMSR-E Radiances in the NCEP Global Data Assimilation System Monthly Weather Review,Vol. 136, no. 2, pp Feb Zhu, Y; Gelaro, R, Observation Sensitivity Calculations Using the Adjoint of the Gridpoint Statistical Interpolation (GSI) Analysis System, Monthly Weather Review. Vol. 136, no. 1, pp Jan DTC GSI documentation ( 22

23 DTC comments GSI is an operation system Well-tested for operation applications Very strict data QC Advanced operator for radiance Bias correction is very important radiance data analysis Model top and channel selection Data coverage for regional application PrepBUFR and NCEP QC procedure The operation system supported by DTC for community users 23

24 DTC and Community GSI Who is DTC Community GSI: Goal and Building Community GSI repository Test and evaluation GSI review committee 24

25 Who is DTC ? Developmental Testbed Center (DTC) Bridge between research and operational community NWP systems supported by DTC WRF, HWRF, GSI, MET, … 25

26 Community GSI: Goals Provide current operation GSI capability to research community (O2R) Provide a pathway for research community to contribute operation GSI (R2O) Provide a framework to enhance the collaboration from distributed GSI developers Provide rational basis to operational centers and research community for enhancement of data assimilation systems 26

27 Community GSI: Building O2R: Code portability: well-tested GSI code package for release Documentation: GSI User’s Guide GSI User’s Webpage Annual release On-site Tutorial (lectures and Practical cases) Help desk R2O: Repository: Build, maintain, sync, test Commit community contributions GSI Review Committee The important role of GSI repository and review committee 27

28 GSI Code Repository GSI Trunk Branch Tag Branch GDAS NDAS Community release HWRF Use tags or branches for: Release, new development, bug fix … Applications may use different revisions in the trunk (“snapshot”). Which GSI should I use ? There is no “DTC GSI”, “EMC GSI” or “global GSI”. There is only one GSI! For a researcher, community release should be sufficient to use. If you are interested in getting new development back to the GSI trunk, contact GSI helpdesk get access to the developmental version of GSI. 28

29 GSI Review Committee GSI Trunk Branch Tag Branch NCEP/E MC NASA/ GMAO NOAA/ ESRL NESDIS DTC AFWA NCAR/ MMM DTC is working as the pathway between the research community and operational community. Provide visitor program for potential contributions to the operational code. Provide equivalent-operational tests of community contributions and provide rational basis for operational implementations. Represent research and operational community Coordinate distributed GSI development Conduct GSI Code Review GSI Review Committee 29

30 History June, 2006:North American Mesoscale (NAM) System, NCEP May, 2007: Global Forecast System (GFS), NCEP 2007: DTC started to document GSI. 2009: Created GSI code repository over NCEP and DTC First GSI release V1.0 Started user support 2010: First Community GSI tutorial Created GSI Review Committee 2011 First Community GSI workshop 30

31 31

32 Available GSI resources for community: O2R User’s Webpage Documentation Annual release package Annual residential tutorial Help desk 32

33 Community GSI – User’s Page Support mainly through User’s Page and help desk: 33

34 Community GSI Release and Tutorial The GSI User’s Guide and the on-line tutorial match with official release code Summer 2013 GSI Tutorial: NCEP, Aug. 5-7, 2013 Summer 2013 GSI Tutorial: NCEP, Aug. 5-7,

35 Community GSI Release 35 Source code and fixed files were based on: the GSI EMC trunk r19180 (May 10, 2012) the community GSI trunk r826 The GSI User’s Guide and the on-line tutorial cases are updated with each official release.

36 Community GSI - Documents User’s Guide Matching with each official release Tutorial lectures Code browser Calling tree Publications 36

37 Community GSI- User’s Guide V3.2 37

38 Community GSI- User’s Guide V3.2 38

39 Community GSI - Practice On-line tutorial for each release Residential tutorial practice cases 39

40 Community GSI – tutorial and workshop 40

41 Community GSI - Help desk Any GSI related questions Most of questions were answered by DTC staff Forward complex questions to GSI developers 41

42 Available GSI resources for community: R2O Commit the community contributions to trunk Help desk 42

43 GSI R2O Transition Procedure Community research Code development candidate 1 1 Code commitment candidate (Branches) Code in repository trunk 1.GSI Review Committee Scientific Review 2.Development, testing and merging 3.GSI Review Committee code and commitment review (2011 Implementation) 43

44 GSI R2O applications DTC contributes to R2O transition, coordinate & assist code commitment: 1. Mariusz Pagowski: Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model 2. DTC: Portability issues from trunk head testing 3. Mariusz Pagowski: Add WRF-Chem model file as background for PM2.5 assimilation 4. Tom Augline: Add control and state variables for Cloudy analysis (triggered more work on the bundles of GMAO) 5. Zhiquan Liu: aerosol work has been merged to the trunk 6. GSD: Recent GSI developments for RR 44

45 Collaboration HFIP program: BUFR processing training AMB branch: GSI application for Rapid Refresh Mariusz Pagowski (AMB): Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model Don Birkenheuer (FAB): Thorpex work using GSI EMC GSI group (feedback from community GSI users) NCAR/MMM DA group (hybrid and cloud analysis,, chemistry analysis ) Space Weather Prediction Center /NOAA (Houjun Wang ) (whole atmosphere data assimilation using GSI) Other GSI users (OU regional ENKF, PSD (Jeff Whitaker) global ENKF …) 45

46 R2O Summary DTC also works with researchers to bring research community contributions back to the GSI operation repository (R2O) Create Trac to keep community researchers in developing loop Help researchers to review and merge the code on top of the trunk Test the code with regression test suite and initial the code review for committing Send your questions to 46

47 Current and future development Community GSI Operational GSI August 9th, 2013 Committee Meeting Agenda 8:45 GMAO 9:30 NESDIS 10:15 break 10:25 ESRL 11:10 NCAR 11:55 lunch 12:40 DTC 1:10 NCEP 3:00 Adjourn 47

48 Community GSI Maintaining the current support GSI review committee Community GSI trunk: sync, test Annual release with code and updated document Improving GSI user’s Website Holding on-site tutorial Maintaining Helpdesk Help committing community contributions Application tools and new instructions to GSI (bundle, hybrid, …) Build operation-similar test system (ongoing) 48

49 Ongoing and future EMC GSI development work EMC GSI development team Update Aug. 8,

50 Use of observations Prepare for METOP-B and CrIS implementation - Andrew Collard Implementation Aug. 20 to WCOSS Includes SEVIRI from Meteosat-10 Use of AURA MLS ozone profiles – Haixia Liu Testing of new version of real time data underway Cloudy Radiance Assimilation – Emily Liu, Yanqiu Zhu, Z. Zhang, Will McCarty (GMAO) Updates to trunk – Significant progress – Testing without convection Improved structure of radiance diagnostic files – Andrew Collard - TBD Satellite wind usage upgrade –XiuJuan Su GOES hourly wind project Ground based Doppler radar data – Shun Liu Conversion to dual-pol Aircraft based Doppler radar data – Mingjing Tong In operations on WCOSS SSMI/S data – Andrew Collard, Emily Liu, and Ellen Liang (NESDIS) To be included in Spring 2014 implementation Mesoscale surface observation – Manuel Pondeca, Steve Levine GLERL surface reduction for lakes under testing GOES-R/SEVERI radiances – Haixia Liu Additional quality control under development Channels used above low clouds CRTM – Paul VanDelst, David Groff, Quanhua Liu(NESDIS), Yong Chen(NESDIS) Improvements continue Improve use of significant levels in RAOB profiles – Wan-shu Wu Bug fixed 50

51 Assimilation techniques Hybrid for NAM – Wan-shu Wu Work continues – results using global ensemble encouraging Hybrid for HWRF – Mingjing Tong HWRF implementation using global ensemble on WCOSS Regional Strong Constraint – David Parrish Cloudy Radiance Assimilation – MinJeong Kim, Emily Liu, Yanqiu Zhu, X. Zhang, Will McCarty(GMAO) See previous Add 4d capability to hybrid – Daryl Kleist Included in trunk – Experimentation now showing significant impact Add precip and cloud physics to strong constraint – Min-Jeong Kim, Daryl Kleist Hurricane location ensemble in hybrid – Yoichiro Ota – Russ Treadon 51

52 Maintenance and general improvements techniques Maintenance of EMC GSI trunk - Mike Lueken Continues many upgrades completed Improvement of RTMA application – Manuel Pondeca Continues Improvement of NAM analysis – Wan-shu Wu Continues Improvement of GFS analysis – Russ Treadon/Andrew Collard Continues – new resolutions and new observations Radiance monitoring – Ed Safford Included in trunk and operations Unification of data monitoring – Ed Safford Underway FOV updates for new satellite instruments – George Gayno, Ninghai Sun (NESDIS) Code Structure – Mike Lueken, Ricardo Todling (GMAO) Transfer to different machines – Daryl Kleist, Russ Treadon, Others WCOSS transition completed – code much more general 52

53 Additional projects beginning for Sandy Supplemental DA Focuses Clouds and precipitation Aircraft data Model parameterizations Increased resolution 4d Hybrid Improved Ensembles 53

54 Final comments 54

55 Advantages of community GSI GSI is a well-supported communalized operation data assimilation system Many groups (EMC, GMAO, GSD, MMM,…) are actively working on various new developments for several operation applications (GFS, NMMB, RR, RTMA, …) DTC has built good expertise to support GSI Has a well maintained repository to make the latest code available to friendly users DTC welcome collaboration and support R2O Users play an important role in improving the support 55

56 Challenges GSI is a fast evolving system Hard to follow for some researchers Need to build new expertise for DTC staff Limited resources to support GSI adds many new functions each year Users application areas are extending Reduced funding for support All support staff are shared with other projects Collaboration are important but cost 56

Download ppt "Ming Hu * Developmental Testbed Center, NCAR-NOAA/GSD **Cooperative Institute for Research in Environmental Sciences, Colorado University at Boulder 在科研中更好的使用美国业务资料分析系统."

Similar presentations

Ads by Google