Presentation is loading. Please wait.

Presentation is loading. Please wait.

Robin Hogan Julien Delanoe Department of Meteorology, University of Reading, UK Variational methods for retrieving cloud, rain and hail properties combining.

Similar presentations

Presentation on theme: "Robin Hogan Julien Delanoe Department of Meteorology, University of Reading, UK Variational methods for retrieving cloud, rain and hail properties combining."— Presentation transcript:

1 Robin Hogan Julien Delanoe Department of Meteorology, University of Reading, UK Variational methods for retrieving cloud, rain and hail properties combining radar, lidar and radiometers

2 Outline Increasingly in active remote sensing, many instruments are being deployed together, and individual instruments may measure many variables –We want to retrieve an optimum estimate of the state of the atmosphere that is consistent with all the measurements –But most algorithms use at most only two instruments/variables and dont take proper account of instrumental errors The variational approach (a.k.a. optimal estimation theory) is standard in data assimilation and passive sounding, but has only recently been applied to radar retrieval problems –It is mathematically rigorous and takes full account of errors –Straightforward to add extra constraints and extra instruments In this talk, two applications will be demonstrated –Polarization radar retrieval of rain rate and hail intensity –Retrieving cloud microphysical profiles from the A-train of satellites (the CloudSat radar, the Calipso lidar and the MODIS radiometer)

3 Polarization radar: Z We need to retrieve rain rate for accurate flood forecasts Conventional radar estimates rain-rate R from radar reflectivity factor Z using Z=aR b –Around a factor of 2 error in retrievals due to variations in raindrop size and number concentration –Attenuation through heavy rain must be corrected for, but gate-by- gate methods are intrinsically unstable –Hail contamination can lead to large overestimates in rain rate

4 Polarization radar: Z dr Differential reflectivity Z dr is a measure of drop shape, and hence drop size: Z dr = 10 log 10 (Z H /Z V ) –In principle allows rain rate to be retrieved to 25% –Can assist in correction for attenuation But –Too noisy to use at each range-gate –Needs to be accurately calibrated –Degraded by hail ZVZV ZHZH Drop Z DR = 0 dB (Z H = Z V ) 1 mm 3 mm 4.5 mm Z DR = 1.5 dB (Z H > Z V ) Z DR = 3 dB (Z H >> Z V ) Drop shape is directly related to drop size: larger drops are less spherical Hence the combination of Z and Z DR can provide rain rate to ~25%.

5 Polarization radar: dp phase shift Differential phase shift dp is a propagation effect caused by the difference in speed of the H and V waves through oblate drops –Can use to estimate attenuation –Calibration not required –Low sensitivity to hail But –Need high rain rate –Low resolution information: need to take derivative but far too noisy to use at each gate: derivative can be negative! How can we make the best use of the Z dr and dp information?

6 Using Z dr and dp for rain Useful at low and high R Differential attenuation allows accurate attenuation correction but difficult to implement Z dr Calibration not required Low sensitivity to hail Stable but inaccurate attenuation correction Need high R to use Must take derivative: far too noisy at each gate Need accurate calibration Too noisy at each gate Degraded by hail dp

7 Variational method Start with a first guess of coefficient a in Z=aR 1.5 Z/a implies a drop size: use this in a forward model to predict the observations of Z dr and dp –Include all the relevant physics, such as attenuation etc. Compare observations with forward-model values, and refine a by minimizing a cost function: Observational errors are explicitly included, and the solution is weighted accordingly For a sensible solution at low rainrate, add an a priori constraint on coefficient a + Smoothness constraints

8 How do we solve this? The best estimate of x minimizes a cost function: At minimum of J, dJ/dx=0, which leads to: –The least-squares solution is simply a weighted average of m and b, weighting each by the inverse of its error variance Can also be written in terms of difference of m and b from initial guess x i : Generalize: suppose I have two estimates of variable x : –m with error m (from measurements) –b with error b (background or a priori knowledge of the PDF of x )

9 The Gauss-Newton method We often dont directly observe the variable we want to retrieve, but instead some related quantity y (e.g. we observe Z dr and dp but not a ) so the cost function becomes –H(x) is the forward model predicting the observations y from state x and may be complex and non-analytic: difficult to minimize J Solution: linearize forward model about a first guess x i –The x corresponding to y=H(x), is equivalent to a direct measurement m : …with error: x y xixi x i+1 x i+2 Observation y (or m )

10 Substitute into prev. equation: –If it is straightforward to calculate y/ x then iterate this formula to find the optimum x If we have many observations and many variables to retrieve then write this in matrix form: –The matrices and vectors are defined by: State vector, a priori vector and observation vector The Jacobian Error covariance matrices of observations and background Where the Hessian matrix is

11 Finding the solution New ray of data First guess of x Forward model Predict measurements y and Jacobian H from state vector x using forward model H(x) Compare measurements to forward model Has the solution converged? 2 convergence test Gauss-Newton iteration step Predict new state vector: x i+1 = x i +A -1 {H T R -1 [y-H(x i )] +B -1 (b-x i )} where the Hessian is A=H T R -1 H+B -1 Calculate error in retrieval The solution error covariance matrix is S=A -1 No Yes Proceed to next ray –In this problem, the observation vector y and state vector x are:

12 Observations Retrieval Forward-model values at final iteration are essentially least- squares fits to the observations, but without instrument noise Chilbolton example 3-GHz radar 25-m dish Chilbolton example 3-GHz radar 25-m dish

13 A ray of data Z dr and dp are well fitted by the forward model at the final iteration of the minimization of the cost function The scheme also reports the error in the retrieved values Retrieved coefficient a is forced to vary smoothly –Represented by cubic spline basis functions

14 Enforcing smoothness 1 Cubic-spline basis functions –Let state vector x contain the amplitudes of a set of basis functions –Cubic splines ensure that the solution is continuous in itself and its first and second derivatives –Fewer elements in x more efficient! Forward model Convert state vector to high resolution: x hr =Wx Predict measurements y and high-resolution Jacobian H hr from x hr using forward model H(x hr ) Convert Jacobian to low resolution: H=H hr W Representing a 50-point function by 10 control points The weighting matrix

15 Enforcing smoothness 2 Background error covariance matrix –To smooth beyond the range of individual basis functions, recognise that errors in the a priori estimate are correlated –Add off-diagonal elements to B assuming an exponential decay of the correlations with range –The retrieved a now doesnt return immediately to the a priori value in low rain rates Kalman smoother in azimuth –Each ray is retrieved separately, so how do we ensure smoothness in azimuth as well? –Two-pass solution: First pass: use one ray as a constraint on the retrieval at the next (a bit like an a priori) Second pass: repeat in the reverse direction, constraining each ray both by the retrieval at the previous ray, and by the first-pass retrieval from the ray on the other side

16 Nominal Z dr error of ±0.2 dB Additional random error of ±1 dB Response to observational errors

17 What if we use only Z dr or dp ? Very similar retrievals: in moderate rain rates, much more useful information obtained from Z dr than dp Z dr only dp only Z dr and dp Retrieved aRetrieval error Where observations provide no information, retrieval tends to a priori value (and its error) dp only useful where there is appreciable gradient with range

18 Observations Retrieval Difficult case: differential attenuation of 1 dB and differential phase shift of 80º Heavy rain and hail

19 How is hail retrieved? Hail is nearly spherical –High Z but much lower Z dr than would get for rain –Forward model cannot match both Z dr and dp First pass of the algorithm –Increase error on Z dr so that rain information comes from dp –Hail is where Z dr fwd -Z dr > 1.5 dB and Z > 35 dBZ Second pass of algorithm –Use original Z dr error –At each hail gate, retrieve the fraction of the measured Z that is due to hail, as well as a. –Now the retrieval can match both Z dr and dp

20 Distribution of hail –Retrieved rain rate much lower in hail regions: high Z no longer attributed to rain –Can avoid false-alarm flood warnings –Use Twomey method for smoothness of hail retrieval Retrieved aRetrieval errorRetrieved hail fraction

21 Enforcing smoothness 3 Twomey matrix, for when we have no useful a priori information –Add a term to the cost function to penalize curvature in the solution: d 2 x/dr 2 (where r is range and is a smoothing coefficient) –Implemented by adding Twomey matrix T to the matrix equations

22 Summary New scheme achieves a seamless transition between the following separate algorithms: –Drizzle. Z dr and dp are both zero: use a-priori a coefficient –Light rain. Useful information in Z dr only: retrieve a smoothly varying a field (Illingworth and Thompson 2005) –Heavy rain. Use dp as well (e.g. Testud et al. 2000), but weight the Z dr and dp information according to their errors –Weak attenuation. Use dp to estimate attenuation (Holt 1988) –Strong attenuation. Use differential attenuation, measured by negative Z dr at far end of ray (Smyth and Illingworth 1998) –Hail occurrence. Identify by inconsistency between Z dr and dp measurements (Smyth et al. 1999) –Rain coexisting with hail. Estimate rain-rate in hail regions from dp alone (Sachidananda and Zrnic 1987) Hogan (2006, submitted to J. Appl. Meteorol.)

23 The A-train The CloudSat radar and the Calipso lidar were launched on 28 th April 2006 They join Aqua, hosting the MODIS, CERES, AIRS and AMSU radiometers An opportunity to tackle questions concerning role of clouds in climate Need to combine all these observations to get an optimum estimate of global cloud properties

24 13.10 UTC June 18 th Scotland England Lake district Isle of Wight France MODIS RGB composite

25 Scotland England Lake district Isle of Wight France MODIS Infrared window UTC June 18 th

26 Scotland England Lake district Isle of Wight France Met Office rain radar network UTC June 18 th


28 Eastern RussiaJapan Sea of JapanEast China Sea Calipso lidar CloudSat radar Molecular scattering Aerosol from China? Cirrus Mixed-phase altocumulus Drizzling stratocumulus Non-drizzling stratocumulus Rain 7 June km

29 Motivation Why combine radar, lidar and radiometers? –Radar Z D 6, lidar D 2 so the combination provides particle size –Radiances ensure that the retrieved profiles can be used for radiative transfer studies Some limitations of existing radar/lidar ice retrieval schemes (Donovan et al. 2000, Tinel et al. 2005, Mitrescu et al. 2005) –They only work in regions of cloud detected by both radar and lidar –Noise in measurements results in noise in the retrieved variables –Elorantas lidar multiple-scattering model is too slow to take to greater than 3rd or 4th order scattering –Other clouds in the profile are not included, e.g. liquid water clouds –Difficult to make use of other measurements, e.g. passive radiances –Difficult to also make use of lidar molecular scattering beyond the cloud as an optical depth constraint –Some methods need the unknown lidar ratio to be specified A unified variational scheme can solve all of these problems

30 Why not to invert the lidar separately Standard method: assume a value for the extinction-to- backscatter ratio, S, and use a gate-by-gate correction –Problem: for optical depth >2 is excessively sensitive to choice of S –Exactly the same instability identified for radar in 1954 Better method (e.g. Donovan et al. 2000): retrieve the S that is most consistent with the radar and other constraints –For example, when combined with radar, it should produce a profile of particle size or number concentration that varies least with range Implied optical depth is infinite

31 Formulation of variational scheme Observation vector State vector –Elements may be missing –Logarithms prevent unphysical negative values Attenuated lidar backscatter profile Radar reflectivity factor profile (on different grid) Ice visible extinction coefficient profile Ice normalized number conc. profile Extinction/backscatter ratio for ice Visible optical depth Aerosol visible extinction coefficient profile Liquid water path and number conc. for each liquid layer Infrared radiance Radiance difference

32 Solution method Find x that minimizes a cost function J of the form J = deviation of x from a-priori + deviation of observations from forward model + curvature of extinction profile New ray of data Locate cloud with radar & lidar Define elements of x First guess of x Forward model Predict measurements y from state vector x using forward model H(x) Also predict the Jacobian H Has solution converged? 2 convergence test Gauss-Newton iteration step Predict new state vector: x i+1 = x i +A -1 {H T R -1 [y-H(x i )] -B -1 (x i -x a )-Tx i } where the Hessian is A=H T R -1 H+B -1 +T Calculate error in retrieval No Yes Proceed to next ray

33 Radar forward model and a priori Create lookup tables –Gamma size distributions –Choose mass-area-size relationships –Mie theory for 94-GHz reflectivity Define normalized number concentration parameter –The N 0 that an exponential distribution would have with same IWC and D 0 as actual distribution –Forward model predicts Z from extinction and N 0 –Effective radius from lookup table N 0 has strong T dependence –Use Field et al. power-law as a-priori –When no lidar signal, retrieval relaxes to one based on Z and T (Liu and Illingworth 2000, Hogan et al. 2006) Field et al. (2005)

34 Lidar forward model: multiple scattering 90-m footprint of Calipso means that multiple scattering is a problem Elorantas (1998) model –O (N m /m !) efficient for N points in profile and m-order scattering –Too expensive to take to more than 3rd or 4th order in retrieval (not enough) New method: treats third and higher orders together –O (N 2 ) efficient –As accurate as Eloranta when taken to ~6th order –3-4 orders of magnitude faster for N =50 (~ 0.1 ms) Hogan (2006, Applied Optics, in press). Code: Ice cloud Molecules Liquid cloud Aerosol Narrow field-of-view: forward scattered photons escape Wide field-of- view: forward scattered photons may be returned


36 Radiance forward model MODIS solar channels provide an estimate of optical depth –Only very weakly dependent on vertical location of cloud so we simply use the MODIS optical depth product as a constraint –Only available in daylight MODIS, Calipso and SEVIRI each have 3 thermal infrared channels in atmospheric window region –Radiance depends on vertical distribution of microphysical properties –Single channel: information on extinction near cloud top –Pair of channels: ice particle size information near cloud top Radiance model uses the 2-stream source function method –Efficient yet sufficiently accurate method that includes scattering –Provides important constraint for ice clouds detected only by lidar –Ice single-scatter properties from Anthony Barans aggregate model –Correlated-k-distribution for gaseous absorption (from David Donovan and Seiji Kato)

37 Ice cloud: non-variational retrieval Donovan et al. (2000) algorithm can only be applied where both lidar and radar have signal Observations State variables Derived variables Retrieval is accurate but not perfectly stable where lidar loses signal Donovan et al. (2000) Aircraft- simulated profiles with noise (from Hogan et al. 2006)

38 Variational radar/lidar retrieval Noise in lidar backscatter feeds through to retrieved extinction Observations State variables Derived variables Lidar noise matched by retrieval Noise feeds through to other variables

39 …add smoothness constraint Smoothness constraint: add a term to cost function to penalize curvature in the solution (J = i d 2 i /dz 2 ) Observations State variables Derived variables Retrieval reverts to a-priori N 0 Extinction and IWC too low in radar-only region

40 …add a-priori error correlation Use B (the a priori error covariance matrix) to smooth the N 0 information in the vertical Observations State variables Derived variables Vertical correlation of error in N 0 Extinction and IWC now more accurate

41 …add visible optical depth constraint Integrated extinction now constrained by the MODIS-derived visible optical depth Observations State variables Derived variables Slight refinement to extinction and IWC

42 …add infrared radiances Better fit to IWC and r e at cloud top Observations State variables Derived variables Poorer fit to Z at cloud top: information here now from radiances

43 Convergence The solution generally converges after two or three iterations –When formulated in terms of ln( ), ln( ) rather than the forward model is much more linear so the minimum of the cost function is reached rapidly

44 Radar-only retrieval Retrieval is poorer if the lidar is not used Observations State variables Derived variables Profile poor near cloud top: no lidar for the small crystals Use a priori as no other information on N 0

45 Radar plus optical depth Note that often radar will not see all the way to cloud top Observations State variables Derived variables Optical depth constraint distributed evenly through the cloud profile

46 Radar, optical depth and IR radiances Observations State variables Derived variables

47 Observed 94-GHz radar reflectivity Observed 905-nm lidar backscatter Forward model radar reflectivity Forward model lidar backscatter Ground-based example Lidar fails to penetrate deep ice cloud

48 Retrieved extinction coefficient Retrieved effective radius r e Retrieved normalized number conc. parameter N 0 Error in retrieved extinction Lower error in regions with both radar and lidar Radar only: retrieval tends towards a-priori

49 Conclusions and ongoing work Variational methods have been described for retrieving cloud, rain and hail, from combined active and passive sensors –Appropriate choice of state vector and smoothness constraints ensures the retrievals are accurate and efficient –Could provide the basis for cloud/rain data assimilation Ongoing work: cloud –Test radiance part of cloud retrieval using geostationary-satellite radiances from Meteosat/SEVIRI above ground-based radar & lidar –Retrieve properties of liquid-water layers, drizzle and aerosol –Apply to A-train data and validate using in-situ underflights –Use to evaluate forecast/climate models –Quantify radiative errors in representation of different sorts of cloud Ongoing work: rain –Validate the retrieved drop-size information, e.g. using a distrometer –Apply to operational C-band (5.6 GHz) radars: more attenuation! –Apply to other problems, e.g. the radar refractivity method Scotland England Lake district Isle of Wight France

50 Sd sdf Banda Sea An island of Indonesia

51 Antarctic ice sheet Southern Ocean

Download ppt "Robin Hogan Julien Delanoe Department of Meteorology, University of Reading, UK Variational methods for retrieving cloud, rain and hail properties combining."

Similar presentations

Ads by Google