Presentation on theme: "Bias correction in data assimilation"— Presentation transcript:
1 Bias correction in data assimilation Dick Dee and Niels BormannECMWFMeteorological Training CourseData Assimilation and Use of Satellite Data27 April 2009
2 OutlineIntroductionBiases in models, observations, and observation operatorsImplications for data assimilationVariational analysis and correction of observation biasThe need for an adaptive systemVariational bias correction (VarBC)Limitations of VarBCInteraction with model biasAssimilation in the upper stratosphereSummary
3 Model bias: Systematic D+3 Z500 errors in three different models ECMWFMeteo-FranceDWDThe above figure shows systematic Z500 errors at D+3 for three different models (ECMWF, Meteo-France and German Weather Service). The first point to notice is that at D+3 the different models show similar systematic errors. While this is perhaps not too surprisingly for the ECMWF and Meteo-France model (some of the model development is shared), the similarity between the ECMWF and the DWD model is astonishing. Summarizing, the above results show that different models have similar systematic error characteristics.Different models often have similar error characteristicsSee Thomas Jung’s TC lecture for much more detail“Predictability, Diagnostics and Seasonal Forecasting" module
4 Model bias: Seasonal variation in upper-stratospheric model errors T255L60 model currently used for ERA-InterimSummer: Radiation, ozone?0.1hPa(65km)Winter: Gravity-wave drag?40hPa(22km)
5 Observation bias: Radiosonde temperature observations Daytime warm bias dueto radiative heating ofthe temperature sensor(depends on solar elevationand equipment type)Mean temperature anomaliesfor different solar elevationsobserved – ERA-40 backgroundat Saigon (200 hPa, 0 UTC)Bias changes due to change of equipment
6 Observation and observation operator bias: Satellite radiances Monitoring the background departures (averaged in time and/or space):Constant bias (HIRS channel 5)Diurnal bias variation in a geostationary satelliteObs-FG bias [K]nadirhigh zenith angleBias depending on scanposition (AMSU-A ch 7)Air-mass dependent bias (AMSU-A channel 14)Obs-FG bias [K]
7 Observation and observation operator bias: Satellite radiances Monitoring the background departures (averaged in time and/or space):HIRS channel 5 (peaking around600hPa) on NOAA-14 satellite has+2.0K radiance bias against FG.Obs-FG bias [K]Same channel on NOAA-16 satellite hasno radiance bias against FG.Obs-FG bias [K]NOAA-14 channel 5 has an instrument bias.
8 Observation and observation operator bias: Satellite radiances Different bias for HIRS due to different spectroscopy in the radiative transfer model:Obs-FG bias [K]Other common causes for biases in radiative transfer:Bias in assumed concentrations of atmospheric gases (e.g., CO2)Neglected effects (e.g., clouds)Incorrect spectral response function….Channel numberOld spectroscopyNew spectroscopyMETEOSAT-9, 13.4µm channel:Drift in bias due to ice-build up on sensor:Sensor decontaminationObs –FG Bias
9 Implications for data assimilation: Bias problems in a nutshell Observations and observation operators have biases, which may change over timeDaytime warm bias in radiosonde measurements of stratospheric temperature; radiosonde equipment changesBiases in cloud-drift wind data due to problems in height assignmentBiases in satellite radiance measurements and radiative transfer modelsModels have biases, and changes in observational coverage over time may change the extent to which observations correct these biasesStratospheric temperature bias modulated by radiance assimilationThis is especially important for reanalysis (trend analysis)Data assimilation methods are primarily designed to correct small (random) errors in the model backgroundLarge corrections generally introduce spurious signals in the assimilationLikewise, inconsistencies among different parts of the observing system lead to all kinds of problems
10 Most assimilation systems assume unbiased models and unbiased data Implications for data assimilation: The effect of model bias on trend estimatesMost assimilation systems assume unbiased models and unbiased dataBiases in models and/or data can induce spurious trends in the assimilation
11 Surface air temperature anomaly (oC) with respect to 1987-2001 Implications for data assimilation: ERA-40 surface temperatures compared to land-station valuesSurface air temperature anomaly (oC) with respect toBased on ERA-40 model simulation (with SST/sea-ice data)Based on monthly CRUTEM2v data (Jones and Moberg, 2003)Based on ERA-40 reanalysisNorthern hemisphere
12 OutlineIntroductionBiases in models, observations, and observation operatorsImplications for data assimilationVariational analysis and correction of observation biasThe need for an adaptive systemVariational bias correction (VarBC)Limitations of VarBCInteraction with model biasAssimilation in the upper stratosphereSummary
13 Variational analysis and bias correction: A brief review of variational data assimilation Minimisebackground constraint (Jb)observational constraint (Jo)The input xb represents past information propagated by the forecast model(the model background)The input [y – h(xb)] represents the new information entering the system(the background departures - sometimes called the innovation)The function h(x) represents a model for simulating observations(the observation operator)Minimising the cost function J(x) produces an adjustment to the model background based on all used observations(the analysis)
14 Variational analysis and bias correction: Error sources in the input data Minimisebackground constraint (Jb)observational constraint (Jo)Errors in the input [y – h(xb)] arise from:errors in the actual observationserrors in the model backgrounderrors in the observation operatorThere is no general method for separating these different error sourceswe only have data about differencesthere is no true reference in the real worldThe analysis does not respond well to contradictory input informationA lot of work is done to remove biases prior to assimilation:ideally by removing the causein practise by careful comparison against other data
15 The need for an adequate bias model Prerequisite for any bias correction is a good model for the bias (b(x,β)):Ideally, “corrects only what we want to correct”.If possible, bias model is guided by the physical origins of the bias.Usually, bias models are derived empirically from observation monitoring.Constant bias (HIRS channel 5)Diurnal bias variation in a geostationary satelliteObs-FG bias [K]Air-mass dependent bias (AMSU-A ch 10)nadirhigh zenith angleBias depending on scanposition (AMSU-A ch 7)1.71.0Obs-FG bias [K]0.0-1.0
16 Past* scheme for radiance bias correction at ECMWF Scan bias and air-mass dependent bias for each sensor/channel were estimated off-line frombackground departures, and stored on files (Harris and Kelly 2001)Error model for brightness temperature data:wherePeriodically estimate scan bias and predictor coefficients:typically 2 weeks of background departures2-step regression procedurecareful masking and data selectionAverage the background departures:Predictors, for instance:hPa thicknesshPa thicknesssurface skin temperaturetotal precipitable water*Replaced in operations September 2006 by VarBC (Variational Bias Correction)
17 The need for an adaptive bias correction system The observing system is increasingly complex and constantly changingIt is dominated by satellite radiance data:biases are flow-dependent, and may change with timethey are different for different sensorsthey are different for different channelsHow can we manage the bias corrections for all these different components?This requires a consistent approach and a flexible, automated system
18 Variational bias correction: The general idea The bias in a given instrument/channel is described by (a few) bias parameters:typically, these are functions of air-mass and scan-position (the predictors)These parameters can be estimated in a variational analysis along with the model state(Derber and Wu, 1998 at NCEP, USA)The standard variational analysis minimizesModify the observation operator to account for bias:Include the bias parameters in the control vector:Minimize insteadWhat is needed to implement this:The modified operator and its TL + adjointA cycling scheme for updating the bias parameter estimatesAn effective preconditioner for the joint minimization problem
19 Variational bias correction: The modified analysis problem The original problem:Jb: background constraintJo: observation constraintJb: background constraint for xJ: background constraint for Jo: bias-corrected observation constraintThe modified problem:Parameter estimatefrom previous analysis
20 Performance: Adaptive bias correction of NOAA-17 HIRS Ch12 p(0): global constantp(1): hPa thicknessp(2): hPa thicknessp(3): surface temperaturep(4): total column watermean analysis departuresmean bias correction
21 Performance: Spinning up new instruments – IASI on MetOp IASI is a high-resolution interferometer with 8461 channelsInitially unstable – data gaps, preprocessing changes
22 Performance: NOAA-9 MSU channel 3 bias corrections (cosmic storm) 200 hPa temperature departures from radiosonde observationsVariational bias correction smoothly handles the abrupt change in bias:initially QC rejects most data from this channelthe variational analysis adjusts the bias estimatesbias-corrected data are gradually allowed back inno shock to the system!
23 Performance: Fit to conventional data Introduction of VarBCin ECMWF operations
24 OutlineIntroductionBiases in models, observations, and observation operatorsImplications for data assimilationVariational analysis and correction of observation biasThe need for an adaptive systemVariational bias correction (VarBC)Limitations of VarBCInteraction with model biasAssimilation in the upper stratosphereSummary
25 Limitations of VarBC: Interaction with model bias VarBC introduces some extra degrees of freedom in the analysis, to help improve the fit to the (bias-corrected) observations:This works well wherethe analysis is well-constrained by observations, and“anchoring” observations are available (e.g.,radiosondes, GPSRO data).VarBC will correct any biased observations and produce a consistent consensus analysis.modelabundant observationsThis may lead to undesired effects wheremodel bias is present, andfew observations are available, oronly observations with VarBC are present.VarBC will, over time, force agreement with the model background.modelobservationsVarBC may wrongly attribute model bias to the observations
26 Limitation of VarBC: Interaction with model bias 0.1hPaSSU Ch3weightingfunctionThe model is generally too cold(by as much as 20K in polar winter)This is wrongly interpreted as an SSU warm biasSSU is then “corrected” to agree with the model40hPaProjection of model cold bias onto SSU Ch3 bias model
27 Weak-constraint 4D-Var (Y. Trémolet) Include model error in the control vector:Constraint is determined by Q: for example, stratosphere only
28 SSU Ch3 mean radiance departures – Aug 1993 ERA-InterimERA-Interim + weak constraint
29 SummaryBiases are everywhere:Many observations cannot be usefully assimilated without bias correctionsManual bias correction for satellite data is no longer feasibleBias parameters can be estimated and adjusted during the assimilationVarBC works well in situations wherethere is sufficient redundancy in the data; orthere are no large model biasesSome questions:How best to represent observation bias with a few parameters?Should VarBC be applied to non-radiance data as well?How much fixed (unbiased) information does the system need?How best to handle model bias in data assimilation?
31 Some references and additional information Harris, B. A. and G. Kelly, 2001: A satellite radiance-bias correction scheme for data assimilation. Q. J. R. Meteorol. Soc., 127,Derber, J. C. and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126,Dee, D. P., 2004: Variational bias correction of radiance data in the ECMWF system. Pp in Proceedings of the ECMWF workshop on assimilation of high spectral resolution sounders in NWP, 28 June-1 July 2004, Reading, UKDee, D. P., 2005: Bias and data assimilation. Q. J. R. Meteorol. Soc., 131,Feel free to contact me with questions:
Your consent to our cookies if you continue to use this website.