3 Towards assimilation of cloud radar and lidar Before we assimilate radar and lidar into NWP models it is helpful to first develop variational cloud retrievalsNeed to develop forward models and their adjoints: used by bothRefine microphysical and a-priori assumptionsGet an understanding of information content from observationsProgress in our development of synergistic radar-lidar-radiometer retrievals of clouds:Variational retrieval of ice clouds applied to ground-based radar-lidar and the SEVIRI radiometer (Delanoe and Hogan 2008)Applied to >2 years of A-Train data (Delanoe and Hogan 2010)Fast forward models for radar and lidar subject to multiple scattering (Hogan 2008, 2009; Hogan and Battaglia 2009)With ESA & NERC funding, currently developing a “unified” algorithm for retrieving cloud, aerosol and precipitation properties from the EarthCARE radar, lidar and radiometers; will apply to other platforms
4 Overview Retrieval framework Minimization techniques: Gauss-Newton vs. Gradient DescentResults from CloudSat-Calipso ice-cloud retrievalComponents of unified retrieval: state variables and forward modelsMultiple scattering radar and lidar forward modelMultiple field-of-view lidar retrievalFirst results from prototype unified retrieval
5 Ingredients developed before In progress Not yet developed Retrieval framework1. New ray of data: define state vectorUse classification to specify variables describing each species at each gateIce: extinction coefficient , N0’, lidar extinction-to-backscatter ratioLiquid: extinction coefficient and number concentrationRain: rain rate and mean drop diameterAerosol: extinction coefficient, particle size and lidar ratio3a. Radar modelIncluding surface return and multiple scattering3b. Lidar modelIncluding HSRL channels and multiple scattering3c. Radiance modelSolar and IR channels4. Compare to observationsCheck for convergence6. Iteration methodDerive a new state vectorEither Gauss-Newton or quasi-Newton scheme3. Forward modelNot convergedConvergedProceed to next ray of data2. Convert state vector to radar-lidar resolutionOften the state vector will contain a low resolution description of the profile5. Convert Jacobian/adjoint to state-vector resolutionInitially will be at the radar-lidar resolution7. Calculate retrieval errorError covariances and averaging kernelIngredients developed beforeIn progressNot yet developed
6 Minimizing the cost function Gradient of cost function (a vector)Gauss-Newton methodRapid convergence (instant for linear problems)Get solution error covariance “for free” at the endLevenberg-Marquardt is a small modification to ensure convergenceNeed the Jacobian matrix H of every forward model: can be expensive for larger problems as forward model may need to be rerun with each element of the state vector perturbedand 2nd derivative (the Hessian matrix):Gradient Descent methodsFast adjoint method to calculate xJ means don’t need to calculate JacobianDisadvantage: more iterations needed since we don’t know curvature of J(x)Quasi-Newton method to get the search direction (e.g. L-BFGS used by ECMWF): builds up an approximate inverse Hessian A for improved convergenceScales well for large xPoorer estimate of the error at the end
7 Combining radar and lidar… Variational ice cloud retrieval using Gauss-Newton methodGlobal-mean cloud fractionCloudsat radarRadar misses a significant amount of iceCALIPSO lidarRadar and lidarRadar onlyLidar onlyTarget classificationInsectsAerosolRainSupercooled liquid cloudWarm liquid cloudIce and supercooled liquidIceClearNo ice/rain but possibly liquidGroundDelanoe and Hogan (2008, 2010)
8 Example of mid-Pacific convection Retrieved extinction (m-1)MODIS 11 micron channelCloudSat radarCirrus detected only by lidarMid-level liquid cloudsDeep convection penetrated only by radarHeight (km)CALIPSO lidarHeight (km)Time since start of orbit (s)
9 Evaluation using CERES longwave flux Retrieved profiles containing only ice are used with Edwards-Slingo radiation code to predict outgoing longwave radiation, and compared to CERESCloudSat-Calipso retrieval (Delanoe & Hogan 2010)CloudSat-only retrieval (Hogan et al. 2006)Bias 0.3 W m-2RMS 14 W m-2Bias 10 W m-2RMS 47 W m-2Nicky Chalmers
10 Evaluation of modelsComparison of the IWC distribution versus temperature for July 2006Met Office model has too little spreadECMWF model lacks high IWC values due to snow thresholdNew ECMWF model version remedies this problemDelanoe et al. (2010)
11 Unified algorithm: state variables Proposed list of retrieved variables held in the state vector xState variableRepresentation with height / constraintA-prioriIce clouds and snowVisible extinction coefficientOne variable per pixel with smoothness constraintNoneNumber conc. parameterCubic spline basis functions with vertical correlationTemperature dependentLidar extinction-to-backscatter ratioCubic spline basis functions20 srRiming factorLikely a single value per profile1Liquid cloudsLiquid water contentOne variable per pixel but with gradient constraintDroplet number concentrationOne value per liquid layerRainRain rateCubic spline basis functions with flatness constraintNormalized number conc. NwOne value per profileDependent on whether from melting ice or coallescenceMelting-layer thickness scaling factorAerosolsExtinction coefficientOne value per aerosol layer identifiedClimatological type depending on regionIce clouds follows Delanoe & Hogan (2008); Snow & riming in convective clouds needs to be addedLiquid clouds currently being tackledBasic rain to be added shortly; Full representation laterBasic aerosols to be added shortly; Full representation via collaboration?
12 Forward model components From state vector x to forward modelled observations H(x)...Ice & snowLiquid cloudRainAerosolxAdjoint of radar model (vector)Adjoint of lidar model (vector)Adjoint of radiometer modelGradient of cost function (vector)xJ=HTR-1[y–H(x)]Vector-matrix multiplications: around the same cost as the original forward operationsAdjoint of radiative transfer modelsyJ=R-1[y–H(x)]Ice/radarLiquid/radarRain/radarIce/lidarLiquid/lidarRain/lidarAerosol/lidarIce/radiometerLiquid/radiometerRain/radiometerAerosol/radiometerLookup tables to obtain profiles of extinction, scattering & backscatter coefficients, asymmetry factorRadar scattering profileLidar scattering profileRadiometer scattering profileSum the contributions from each constituentRadar forward modelled obsLidar forward modelled obsRadiometer forward modelled obsH(x)Radiative transfer models
13 Scattering modelsFirst part of a forward model is the scattering and fall-speed modelSame methods typically used for all radiometer and lidar channelsRadar and Doppler model uses another set of methodsGraupel and melting ice still uncertainParticle typeRadar (3.2 mm)Radar DopplerThermal IR, Solar, UVAerosolAerosol not detected by radarMie theory, Highwood refractive indexLiquid dropletsMie theoryBeard (1976)Rain dropsT-matrix: Brandes et al. (2002) shapesIce cloud particlesT-matrix (Hogan et al. 2010)Westbrook & HeymsfieldBaran (2004)Graupel and hailTBDMelting iceWu & Wang (1991)
14 Radiative transfer forward models Computational cost can scale with number of points describing vertical profile N; we can cope with an N2 dependence but not N3Radar/lidar modelApplicationsSpeedJacobianAdjointSingle scattering: b’=b exp(-2t)Radar & lidar, no multiple scatteringNN2Platt’s approximation b’=b exp(-2ht)Lidar, ice only, crude multiple scatteringPhoton Variance-Covariance (PVC) method (Hogan 2006, 2008)Lidar, ice only, small-angle multiple scatteringN or N2Time-Dependent Two-Stream (TDTS) method (Hogan and Battaglia 2008)Lidar & radar, wide-angle multiple scatteringN3Depolarization capability for TDTSLidar & radar depol with multiple scatteringLidar uses PVC+TDTS (N2), radar uses single-scattering+TDTS (N2)Jacobian of TDTS is too expensive: N3We have recently coded adjoint of multiple scattering modelsFuture work: depolarization forward model with multiple scatteringRadiometer modelApplicationsSpeedJacobianAdjointRTTOV (used at ECMWF & Met Office)Infrared and microwave radiancesNTwo-stream source function technique (e.g. Delanoe & Hogan 2008)Infrared radiancesN2LIDORTSolar radiancesInfrared will probably use RTTOV, solar radiances will use LIDORTBoth currently being tested by Julien Delanoe
15 Examples of multiple scattering LITE lidar (l<r, footprint~1 km)CloudSat radar (l>r)StratocumulusIntense thunderstormSurface echoApparent echo from below the surface
16 Fast multiple scattering forward model Hogan and Battaglia (J. Atmos. Sci. 2008)New method uses the time-dependent two-stream approximationAgrees with Monte Carlo but ~107 times faster (~3 ms)Added to CloudSat simulatorCloudSat-like exampleCALIPSO-like example
17 Multiple field-of-view lidar retrieval To test multiple scattering model in a retrieval, and its adjoint, consider a multiple field-of-view lidar observing a liquid cloudWide fields of view provide information deeper into the cloudThe NASA airborne “THOR” lidar is an example with 8 fields of viewSimple retrieval implemented with state vector consisting of profile of extinction coefficientDifferent solution methods implemented, e.g. Gauss-Newton, Levenberg-Marquardt and Quasi-Newton (L-BFGS)lidarCloud top100 m10 m600 m
18 Results for a sine profile Simulated test with 200-m sinusoidal structure in extinctionWith one FOV, only retrieve first 2 optical depthsWith three FOVs, retrieve structure of extinction profile down to 6 optical depthsBeyond that the information is smeared outNicola Pounder
19 Optical depth from multiple FOV lidar Despite vertical smearing of information, the total optical depth can be retrieved to ~30 optical depthsLimit is closer to 3 for one narrow field-of-view lidarNicola Pounder
20 Comparison of convergence rates Solution is identicalGauss-Newton method converges in < 10 iterationsL-BFGS Gradient Descent method converges in < 100 iterationsConjugate Gradient method converges a little slower than L-BFGSEach L-BFGS iteration >> 10x faster than each Gauss-Newton one!Gauss-Newton method requires the Jacobian matrix, which must be calculated by rerunning multiple scattering model multiple times
21 Unified algorithm: first results for ice+liquid But lidar noise degrades retrievalConvergence!TruthRetrievalFirst guessIterationsObservations RetrievalObservationsForward modelled retrievalForward modelled first guess
22 Add smoothness constraint Smoother retrieval but slower convergenceTruthRetrievalFirst guessIterationsObservations RetrievalObservationsForward modelled retrievalForward modelled first guess
23 Unified algorithm: progress Done:Functioning algorithm framework existsC++: object orientation allows code to be completely flexible: observations can be added and removed without needing to keep track of indices to matrices, so same code can be applied to different observing systemsCode to generate particle scattering libraries in NetCDF filesAdjoint of radar and lidar forward models with multiple scattering and HSRL/Raman supportInterface to L-BFGS algorithm in GNU Scientific LibraryIn progress / future work:Debug adjoint code (so far we are using numerical adjoint - slow)Implement full ice, liquid, aerosol and rain constituentsEstimate and report error in solution and averaging kernelInterface to radiance modelsTest on a range of ground-based, airborne and spaceborne instruments, particularly the A-Train and EarthCARE satellitesAssimilation?
Your consent to our cookies if you continue to use this website.