Presentation on theme: "Review of NCEP GFS Forecast Skills and Major Upgrades"— Presentation transcript:
1 Review of NCEP GFS Forecast Skills and Major Upgrades Fanglin YangIMSG - Environmental Modeling CenterNational Centers for Environmental PredictionCamp Springs, Maryland, USA24th Conference on Weather and Forecasting & 20th Conference on Numerical Weather Prediction91th AMS Annual Meeting, 23–27 January 2011, Seattle, WA
2 GFS 500hPa Height Anomaly Correlation, 00Z Cycle Day-5 Forecast GFS forecast skill has been steadily improving in both the Northern and Southern Hemisphere.GFS performs better in NH than in SH, especially in early years.NH score was improved significantly in 2010.
3 Day-5 Northern-Hemisphere 500-hPa Height Anomaly Correlation The forecast skills of all NWP models have been steadily improving.GFS lags behind ECMWF, and is comparable to UKM.The difference between GFS and CDAS is an indicator of improvement in GFS physics, dynamics and data assimilation system.GFS: NCEP Global Forecast System; FNO: Navy Fleet Numerical Meteorology and Oceanography Center; ECM: European Center for Medium-Range Weather Forecasts; UKM: The United Kingdom Met Office; CMC: The Canadian Meteorological Center; CDAS: T version of GFS used for the NCEP/NCAR Reanalysis.
4 Day-5 Southern-Hemisphere 500-hPa Height Anomaly Correlation GFS lags behind ECMWF;UKM surpassed GFS since 2005;CMC improved in the past couple of years.GFS: NCEP Global Forecast System; FNO: Navy Fleet Numerical Meteorology and Oceanography Center; ECM: European Center for Medium-Range Weather Forecasts; UKM: The United Kingdom Met Office; CMC: The Canadian Meteorological Center; CDAS: T version of GFS used for the NCEP/NCAR Reanalysis.
5 Tropical Wind RMSE 850 hPa 200 hPa GFS 850hPa wind has been significantly improved over the years, and is closing to ECMWF forecast. Improvement of 200hPa wind is modest.It is worth noting that RMSE can be misleading if the forecast model is heavily damped.
6 NH SH Each NWP model has its own strength and weakness. For instance, GFS has smaller forecast error than ECMWF in the stratosphere.500-hPa HGT RMSE, Dec 2010500-hPa HGT, Fcst v.s. AnalysisNHSHECMWF develops cold bias in the stratosphereForecast Hour 0-240
7 GFS Annual Mean Day-5 NH 500-hPa Height Anomaly Correlation Day-5 AC has improved by about 0.1 in the past 10 yearsForecasts with AC >= 0.6 is usually regarded as useful.GFS useful forecasts have improved from 6.4 days in 2001 to 8 days in 2010.ECMWF is still about 0.5 day ahead of GFS.
8 GFS Annual Mean Day-5 SH 500-hPa Height Anomaly Correlation SH Day-5 AC has improved by about 0.12 in the past 10 yearsForecasts with AC >= 0.6 is usually regarded as useful.GFS SH useful forecasts have improved from 6.1 days in 2001 to 7.4 days in 2010.ECMWF is about 0.8 day ahead of GFS.
9 GFS Precip Skill Scores Over CONUS, fh60-84 (day-3) ETS increased and BIAS decreased in recent years
10 Precip Skill Scores contingency table: Obs YesObs NOFcst YESabFcst NOcdcontingency table:Hits (a): occasions/counts where both forecast and observation are greater than or equal to a threshold over, say, CONUS;False alarms (b): occasions where forecast is above a threshold whereas observation is under the same threshold;Misses (c): occasions where the observation is above a threshold and forecast is under the same threshold;No forecasts (d): occasions where both forecast and observation are under the threshold.Bias Score: BS=(a + b)/(a + c) measures over-forecasts (BS>1) or under-forecasts (BS<1) precipitation frequency over an area for a selected threshold.Equitable Threat Score: EQ_TS=(a - ar)/(a + b + c - ar) where ar is the expected number of correct forecasts above the threshold in a random forecast where forecast occurrence/non-occurrence is independent from observation/non-observation, ar=(a + b)*(a + c)/(a + b + c + d). EQ_TS=1 means a perfect forecast. EQ_TS <=0 means the forecast is useless.
11 New (SAS, PBL and Shallow Convection) Han & Pan, 2010OBSCTL24 h accumulated precipitation ending at 12 UTC, July 24, 2008 from (a) observation and h forecasts with (b) control GFS and (c) revised modelNew (SAS, PBL and Shallow Convection)Reduce unrealistic excessive heavy precipitation (so called grid-scale storm or bull’s eye precipitation)
12 GFS Hurricane Track and Intensity Forecasts, Atlantic Basin Intensity forecast has been significantly improved, mainly due to increases in model resolution.The current GFS intensity forecast is catching up with HWRF. Forecasters have started to take GFS into account for intensity forecast.Track forecast within 3 days has been steadily improving, although the pace is slow.Beyond day 3, the forecast still varies from year to year.
13 GFS Hurricane Track and Intensity Forecasts, Eastern Pacific Track forecast within 3 days has been steadily improving, although the pace is slow.Intensity forecast not improved in the Eastern Pacific Basin. Why ??
14 What kind of model changes have contributed to the improvement of GFS forecast skills? Does very model upgrade always lead to better forecast skill?
15 GFS Changes 3/1999 AMSU-A and HIRS-3 data 2/2000 The “GFS Changes” slides were first scripted by Stephen LordGFS Changes3/1999AMSU-A and HIRS-3 data2/2000Resolution change: T126L28 T170L42 (100 km 70 km)Next changes7/2000 (hurricane relocation)8/2000 (data cutoff for 06 and 18 UTC)10/2000 – package of minor changes2/2001 – radiance and moisture analysis changes5/2001Major physics upgrade (prognostic cloud water, cumulus momentum transport)Improved QC for AMSU radiances6/2001 – vegetation fraction7/2001 – SST satellite data8/200 – sea ice mask, gravity wave drag adjustment, random cloud tops, land surface evaporation, cloud microphysics…)10/ 2001 – snow depth from model background1/2002 – Quikscat included
16 GFS Changes (cont)11/2002Resolution change: T170L42 T254L64 (70 km 55 km)Recomputed background errorDivergence tendency constraint in tropics turned offNext changes3/2003 – NOAA-17 radiances, NOAA-16 AMSU restored, Quikscat 0.5 degree data8/2003 – RRTM longwave and trace gases10/2003 – NOAA-17 AMSU-A turned off11/2003 – Minor analysis changes2/2004 – mountain blocking added5/2004 – NOAA-16 HIRS turned off5/2005Resolution change: T254L64 T382L64 ( 55 km 38 km )2-L OSU LSM 4-L NOHA LSMReduce background vertical diffusionRetune mountain blocking6/2005 – Increase vegetation canopy resistance7/2005 – Correct temperature error near top of model
17 GFS Changes (cont) 8/2006 5/2007 12/2007 2/2009 7/2010 Revised orography and land-sea maskNRL ozone physicsUpgrade snow analysis5/2007SSI (Spectral Statistical Interpolation) GSI ( Gridpoint Statistical Interpolation).Vertical coordinate changed from sigma to hybrid sigma-pressureNew observations (COSMIC, full resolution AIRS, METOP HIRS, AMSU-A and MHS)12/2007JMA high resolution winds and SBUV-8 ozone observations added2/2009Flow-dependent weighting of background error variancesVariational Quality ControlMETOP IASI observations addedUpdated Community Radiative Transfer Model coefficients7/2010Resolution Change: T382L64 T574L64 ( 38 km 23 km )Major radiation package upgrade (RRTM2 , aerosol, surface albedo etc)New mass flux shallow convection scheme; revised deep convection and PBL schemePositive-definite tracer transport scheme to remove negative water vapor
18 500-hPa Height AC Frequency distribution, GFS 00Z Cycle Day-5 Forecast Look at the history of extremes in the distributionPoor Forecasts (AC < 0.7 )Excellent forecasts ( AC > 0.9 )Twenty bins were used to count for the frequency distribution, with the 1st bin centered at and the last been centered at The width of each bin is 0.05.
19 Percent of Poor Forecasts (AC <0.7) v.s. Model Changes Resolution:3/1991: T80L18 T126L28 (100km)2/2000: T126L28 T170L42 (70km)11/2002: T170L42 T254L64 (55km)6/2005: T254L64 T382L64 (38km)7/2010: T382L64 T574L64 (23km)Physics and Data Assimilation:3/1999: AMSU-A & HIRS-3 data5/2001: prognostic cloud water, cumulus momentum transport6/2005: OSU 2-L LSM to 4-L NOHA LSM5/2007: SSI to GSI; Hybrid sigma-p; New observations2/2009: flow-dependent error covariance; Variational QC7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.NHA2B34, C5, Fyear
20 Percent of Poor Forecasts (AC <0.7) v.s. Model Changes Resolution:3/1991: T80L18 T126L28 (100km)2/2000: T126L28 T170L42 (70km)11/2002: T170L42 T254L64 (55km)6/2005: T254L64 T382L64 (38km)7/2010: T382L64 T574L64 (23km)Physics and Data Assimilation:3/1999: AMSU-A & HIRS-3 data5/2001: prognostic cloud water, cumulus momentum transport6/2005: OSU 2-L LSM to 4-L NOHA LSM5/2007: SSI to GSI; Hybrid sigma-p; New observations2/2009: flow-dependent error covariance; Variational QC7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.SHA2B34, C5, FDEyear
21 Percent of Excellent Forecasts (AC >0.9) v.s. Model Changes Resolution:3/1991: T80L18 T126L28 (100km)2/2000: T126L28 T170L42 (70km)11/2002: T170L42 T254L64 (55km)6/2005: T254L64 T382L64 (38km)7/2010: T382L64 T574L64 (23km)Physics and Data Assimilation:3/1999: AMSU-A & HIRS-3 data5/2001: prognostic cloud water, cumulus momentum transport6/2005: OSU 2-L LSM to 4-L NOHA LSM5/2007: SSI to GSI; Hybrid sigma-p; New observations2/2009: flow-dependent error covariance; Variational QC7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.NH5, FE4, C3year
22 Percent of Excellent Forecasts (AC >0.9) v.s. Model Changes Resolution:3/1991: T80L18 T126L28 (100km)2/2000: T126L28 T170L42 (70km)11/2002: T170L42 T254L64 (55km)6/2005: T254L64 T382L64 (38km)7/2010: T382L64 T574L64 (23km)Physics and Data Assimilation:3/1999: AMSU-A & HIRS-3 data5/2001: prognostic cloud water, cumulus momentum transport6/2005: OSU 2-L LSM to 4-L NOHA LSM5/2007: SSI to GSI; Hybrid sigma-p; New observations2/2009: flow-dependent error covariance; Variational QC7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.SHED?3year
23 CommentsMost implementations include both major and minor changes. They all contribute to improving the system. Accumulated impact of many small changes is significant but not measurable.Predictability may change from year to year.
24 Testing and evaluation Benefits and remaining issues Most Recent GFS Upgrade 28-July Implementation T382L64 (38 km) T574L64 (23 km) & Major physics upgradeMajor changesTesting and evaluationBenefits and remaining issues
25 Major Changes Resolution and ESMF Radiation and cloud T382L64 to T574L64 ( ~38 km -> ~27 km) for fcst1 (0-192hr) & T190L64 for fcst2 ( hr) .fcst2 step with digital filter turned onESMF 3.1.0rp2Radiation and cloudChanging SW routine from ncep0 to RRTM2Changing longwave computation frequency from three hours to one hourAdding stratospheric aerosol SW and LW and tropospheric aerosol LWChanging aerosol SW single scattering albedo from 0.90 in the operation to 0.99Changing SW aerosol asymmetry factor. Using new aerosol climatology.Changing SW cloud overlap from random to maximum-random overlapUsing time varying global mean CO2 instead of constant CO2 in the operationUsing the Yang et al. (2008) scheme to treat the dependence of direct-beam surface albedo on solar zenith angle over snow-free land surface
26 Example: Improving GFS Surface Albedo Using ARM-SURFRAD Observations Fits using data at ARM and SURFRAD stationsDependencies of direct-beam albedo, normalized by the diffuse albedo, on SZA. The ten colored long-dashed lines represent the empirical fits derived from observations at the three ARM and seven SURFRAD stations for the entire-day cases. The blue line with filled circles is based on the observations at all stations except the Desert Rock station (the line with crosses). The black lines with open circles and squares are governed by the NCEP GFS parameterization with the constant being set to 0.4 and 0.1, respectively.Fanglin Yang, Kenneth Mitchell, Yu-Tai Hou, Yongjiu Dai, Xubin Zeng, Zhou Wang, and Xin-Zhong Liang, 2008: Dependence of land surface albedo on solar zenith angle: observations and model parameterizations. Journal of Applied Meteorology and Climatology. No.11, Vol 47,
27 Major Changes Gravity-Wave Drag Parameterization Using a modified GWD routine to automatically scale mountain block and GWD stress with resolution.Compared to the T382L64 GFS, the T574L64 GFS uses four times stronger mountain block and one half the strength of GWD.Removal of negative water vaporUsing a positive-definite tracer transport scheme in the vertical to replace the operational central-differencing scheme to eliminate computationally-induced negative tracers.Changing GSI factqmin and factqmax parameters to reduce negative water vapor and supersaturation points from analysis step.Modifying cloud physics to limit the borrowing of water vapor that is used to fill negative cloud water to the maximum amount of available water vapor so as to prevent the model from producing negative water vapor.Changing the minimum value of water vapor mass mixing ratio in radiation from 1.0e-5 in the operation to 1.0e-20. Otherwise, the model artificially injects water vapor in the upper atmosphere where water vapor mixing ratio is often below 1.0e-5.
28 Vertical Advection of Tracers: Current GFS Scheme Flux form conserves massCurrent GFS uses central differencing in space and leap-frog in time.The scheme is not positive definite and may produce negative tracers.
29 Example: Removal of Negative Water Vapor Sources of Negative Water VaporVertical advectionData assimilationSpectral transformBorrowing by cloud waterSAS Convection_Ops GFSData AssimilationFlux-Limited Vertically-Filtered Scheme, central in timeData AssimilationNewB: horizontal advection, computed in spectral form with central differencing in spaceA: vertical advection, computed in finite-difference form with flux-limited positive-definite scheme in spacePositive-definiteFanglin Yang et al., 2009: On the Negative Water Vapor in the NCEP GFS: Sources and Solution. 23rd Conference on Weather Analysis and Forecasting/19th Conference on Numerical Weather Prediction, 1-5 June 2009, Omaha, NE
32 Vertical Advection of Tracers: Idealized Case Study windUpwind (diffusive)Flux-LimitedInitial conditionGFS Central-in-Space
33 Summary: Negative Water Vapor in the GFS CausesImportanceSolutionsVertical AdvectionSemi-LagrangianFlux-Limited Positive-Definite Scheme for current Eulerian GFSGSI AnalysisTuning factqmin and factqmaxSpectral Transform1. Semi-Lagrangian GFS: running tracers on grid, no spectral transform2. Eulerian GFS: no solution yet.Cloud Water BorrowingLimiting the borrowing to available amount of water vaporSAS Mass-FluxRemains to be resolved
34 Major Changes New mass flux shallow convection scheme (Han & Pan 2010) Use a bulk mass-flux parameterization same as deep convection schemeSeparation of deep and shallow convection is determined by cloud depth (currently 150 mb)Entrainment rate is given to be inversely proportional to height (which is based on the LES studies) and much smaller than that in the deep convection schemeMass flux at cloud base is given as a function of the surface buoyancy flux (Grant, 2001), which contrasts to the deep convection scheme using a quasi-equilibrium closure of Arakawa and Shubert (1974) where the destabilization of an air column by the large-scale atmosphere is nearly balanced by the stabilization due to the cumulusRevised deep convection scheme (Han & Pan 2010)Random cloud top selection in the current operational scheme is replaced by an entrainment rate parameterization with the rate dependent upon environmental moistureInclude the effect of convection-induced pressure gradient force to reduce convective momentum transport (reduced about half)Trigger condition is modified to produce more convection in large-scale convergent regions but less convection in large-scale subsidence regionsA convective overshooting is parameterized in terms of the convective available potential energy (CAPE)
35 Major Changes Revised Boundary Layer Scheme (Han & Pan 2010) Include stratocumulus-top driven turbulence mixing based on Lock et al.’s (2000) studyEnhance stratocumulus top driven diffusion when the condition for cloud top entrainment instability is metUse local diffusion for the nighttime stable PBL rather than a surface layer stability based diffusion profileBackground diffusivity for momentum has been substantially increased to 3.0 m2s-1 everywhere, which helped reduce the wind forecast errors significantlyHurricane relocationRunning hurricane relocation at the 1760x880 forecast grid instead of the 1152x576 analysis gridPosting GDAS pgb files first on Guassian grid (1760x880), then convert to 0.5-deg for hurricane relocation.
36 Example: New Mass-Flux Based Shallow Convection By Jongil Han and Hua-lu Pan Mass flux analogy (de Roode et al., 2000) :Au (updraft area)=0.5Ad (downdraft area)=0.5Operational shallow convection scheme (Diffusion scheme, Tiedke, 1983)Au~0.0; Ad~1.0Environment is dominated by subsidence resulting in environmental warming and drying.New shallow convection scheme (Mass flux scheme)
37 Heating by Shallow Convection Ops GFSNew shallow convection scheme
39 Low cloud cover (%)No stratocumulus top driven diffusionWith stratocumulus top driven diffusion
40 OBSCTL24 h accumulated precipitation ending at 12 UTC, July 24, 2008 from (a) observation and h forecasts with (b) control GFS and (c) revised modelNewReduce unrealistic excessive heavy precipitation (so called grid-scale storm or bull’s eye precipitation)
41 Parallel Test & Evaluation 2008 Hurricane Season (June 20 – November 10)2009 Hurricane Season (June 20 – November 10)2009/2010 Winter and 2010 Spring (December 1 – present)
42 500hPa Height ACNH 2008NH 2009SH 2009SH 2008Significant improvement in Anomaly correlations for week-one Fcst
44 Precipitation Skill Scores over CONUS 20082009Significantly improved EQ scores, reduced biases for heavy precip events
45 Hurricane Track and Intensity: 2008 Atlantic TrackEast Pacific TrackT382T574Atlantic IntensityEast Pacific IntensityReduced track errors in both basins, significantly improved intensity forecast
46 Hurricane Track and Intensity: 2009 Atlantic TrackEast Pacific TrackAtlantic IntensityEast Pacific IntensityReduced track error in East Pacific,significantly improved intensity forecast in both basins.
48 SummaryThe upcoming T574L64 implementation in July 2010 was a major improvement upon the last operational T382L64 GFS in terms of height AC, wind RMSE, precipitation skill score, and hurricane track and intensity.However, there are still a few remaining issues.
49 QBO transition from westerly phase to easterly phase Ops T382T574ECMWFT382 GFS is closer to ECMWF than the T574 GFS does.T574 GFS has weaker easterly than T382 GFS in 2009 and 2010.This is caused by overly too strong vertical diffusion of momentum in the stratosphere. The spring 2011 GFS implementation will has this problem corrected.
50 Planed Upgrades T878 L64 or L91 Semi- Lagrangian GFS NEMS (a unified national environment modeling system)