1. MPI-Meteorology Hamburg, Germany Evaluation of year 2004 monthly GlobAER aerosol products Stefan Kinne

2. the task an evaluation of  GlobAER 2004 global maps for aerosol optical depth(info on amount) for Angstrom parameter(info on size)  by ASTR (dual view, global, at best 10 per month)  by MERIS (nadir view, global, at best 1 per day)  by SEVIRI (nadir view, regional, at best 30 per day)  by a merged (ATSR / MERIS / SEVIRI) composite

3. the questions  how well do the data compare to trusted data references (e.g. AERONET) ?  how well do the data compare to existing data sets – even for same sensor data ?  can the performance be quantified ? more specifically …  what are the scores of a new (outlier- resistant) method … examining bias, spatial and temporal variability ?

4. the investigated properties  aerosol optical depth (AOD) extinction along a (vertical) direction due to scattering and absorption by aerosol  here for the entire atmosphere  here for the mid-visible (0.55  m wavelength)  Angstrom parameter (Ang) spectral dependence of AOD in the visible spectrum  small dependence (Ang ~ 0)  aerosol > 1  m size  strong decrease (Ang > 1.2)  aerosol < 0.5  m size

5. monthly data-sets GlobAER 2004 maps  GAa - ATSR  GAs - SEVIRI  GAm - MERIS  GAx – merged other multi-ann. maps  med – model median  clim – med & aer(sun)  sky – med & aer(sky)  TO – TOMS other 2004 maps  ATs - ATSR Swansea  SEb- SEVIRI Bruxelles  Mdb - MODIS deep blu  MO - MODIS std coll.5  MI – MISR version 22  Ag – AVHRR, GACP  Ap – AVHRR, Patmos  Aer – AERONET 2004

6. AOD map comparisons  AOD annual maps all available data  AOD seasonal maps ATSR GlobAER vs Swansey SEVIRI GloabAER vs RUIB-Bruexelles  difference maps … to a remote sensing ‘best’ composite all available data-sets focus on the four GloabAER products

7. AOD – 2004 annual maps

8. ATSR / SEVIRI – seasonal AOD

9. AOD diff to ‘composite’  underestimates overestimates 

10. quick (annual) AOD check  ATSR underestimates in dust regions overestimate in biomass regions  SEVIRI severe biomass overestimates useful over-land estimates ?  MERIS apparent land snow cover issue  merged not the envisioned improvement  ‘-’ ’+’ 

11. Angstrom map comparisons  Angstrom annual maps all available data  Angstrom seasonal maps ATSR GlobAER vs Swansey SEVIRI GloabAER vs RUIB-Bruexelles  difference maps … to a climatology (model & AERONET) all available data-sets focus on the four GloabAER products

12. Angstrom – 2004 annual maps

13. ATSR / SEVIRI – seasonal Angstr.

14. Angstrom diff to ‘climatology’  underestimates overestimates ....

15. quick annual Angstrom check  ATSR underestimates in tropics overstimates in south. oceans  SEVIRI underestimates over oceans strong overestimates over land  MERIS overestimates over land  merged not the envisioned improvement  ‘-’ ’+’ 

16. the SCORING challenge  quantify data performance by one number  develop a score such that contributing errors to be traceable back to bias spatial correlation temporal correlation spatial sub-scale (e.g. region) temporal sub-scale (e.g. month, day)  make this score outlier resistant

17. one number ! - 0.504

18. info on overall bias - 0.504 sign of the bias

19. | 1 | is perfect …. 0 is poor - 0.504 sign of the bias the closer to absolute 1.0 … the better

20. product of sub-scores - 0.504 = 0.9 *- 0.7 * 0.8 the closer to absolute 1.0 … the better temporal correlation sub-score bias sub- score spatial correlation sub-score sign of the bias

21. spatial stratification - 0.504 = 0.9 * -0.7 * 0.8 time score bias score spatial score spatial sub-scale scores overall score regional surface area weights TRANSCOM regions

22. temporal stratification - 0.504 = 0.9 * -0.7 * 0.8 time score bias score spatial score spatial sub-scale scores overall score temporal sub-scale scores (e.g. month or days) averaging in time instantaneous median data

23. sub-score definition  each sub-score S is defined by an error e and by an error weight w 0.9 * -0.7 * 0.8 S = 1 – w * e time score S bias score S spatial score S spatial sub-scale scores temporal sub-scale scores (e.g. month or days) instantaneous median data

24. definition of errors e  S = 1 – w * e   all values for the errors e are rank - based for “time score” and “spatial score” rank correlation coefficients for data pairs are determined  e, correlation = (1- rank_correlation coeff.) /2 (correlated: e = 0, anti-correlated: e = 1) time score bias score spatial score

25. definition of errors e  S = 1 – w * e for “bias score” all all data-pairs are placed in a single array and ranked by value then ranks are separated according to data origin, summed and (rank-sums are) compared  e, bias = (sum1 – sum2) / (sum1 + sum2) (strong neg.bias e = -1, strong pos.bias e= +1) an example (“how does the rank bias error work ?”)  set 1: 1 7 8 value: 9 8 7 4 3 1 rank-sum 1: 11  set 2: 3 4 9 rank: 1 2 3 4 5 6 rank-sum 2: 10 e = (1-2)/(1+2) = (11-10)/21 ~zero  no clear bias time score bias score spatial score

26. definition of error weight w  S = 1 – w * e  w is a weight factor based on the inter-quartile range / median ratio w = (75%pdf - 25%pdf) / 50%pdf … but not larger than 1.0 (w<1.0) simply put … if there is no variability an error does not matter bias score spatial score time score

27. scoring summary  one single score …  … without sacrificing spatial and temporal detail !  stratification into error contribution from bias spatial correlation temporal correlation  robustness against outliers still … just one of many possible approaches  now to some applications …

28. questions  how did GlobAER products score? overall ? seasonality ? spatial correlation ? bias ? in what regions ? in what months ?  how did scores place to other retrievals … with the same sensor (for the same year 2004) with other sensors (for the same year 2004)

29. selective evaluations for year 2004 data ..atsrSwansey ATSR ..gaatGlobAER ATSR 3/2009 (std) ..gaa2GlobAER ATSR 7/2009 (filtered) ..gaa3GlobAER ATSR8/2009 (test) ..misrMISR ver.22  how does ‘gaa2’ score ?  score diff between ‘gaa2’ and ‘misr’  score diff between ‘gaa2’ and ‘gaat’  score diff between ‘gaa2’ and ‘atsr’

30. performance vs AERONET GlobAER ATSR … “gaat” AOD score:.47 ANG score: -.57 Swansey ATSR … “atsr” AOD score:.57 ANG score: -.49 GlobAER ATSR (filter) … “gaa2” AOD score: none * ANG score: none * MISR v22 AOD score:.59 ANG score:.67 * too few areas with scores

31. side by side let us compare AOD details …  what is the sign of the bias ?  what is the bias strength (median diff) ?  what is the bias error ?  what is the spatial variability error ?  what is the seasonality error ?  what is the overall error  what are selected error differences among different retrieval products?

32. AOD bias sign vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22 overestimateunderestimate ATSR GlobAER filtered

33. AOD bias error vs AERONET ATSR GlobAER ATSR Swansey MODIS coll.5 MISR vers.22

34. AOD bias strength vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22

35. AOD bias error vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22 ATSR GlobAER filtered

36. AOD spatial var. Vs. AERONET ATSR GlobAER ATSR Swansey MISR vers.22 ATSR GlobAER, filtered

37. AOD season error vs AERONET ATSR GlobAER ATSR Swansey MISR vers.22

38. AOD total error vs AERONET ATSR Swansey MISR vers.22 ATSR GlobAER filtered ATSR GlobAER

39. AOD ATSR GlobAER filtered - total error

40. AOD ATSR GlobAER filtered - bias error

41. AOD ATSR GlobAER filtered - spatial error

42. AOD summary  ATSR products ATSR is most promising (merged is poorer!)  ATSR AOD of GLOBAER poorer than the ATSR AOD by Swansey much poorer than MODIS or MISR filtered version scores poorer mainly due to deterioation in spatial and temp. variability  ATSR AOD by GlobAER with filter bias error is reasonable spatial variability is poor seasonality is poor

43. extras  score differences … for overall errors neg. difference  smaller error pos. difference  larger error  gaa2 vs atsr  gaat vs atsr  gaa2 vs gaat  gaat vs misr

44. gaa2 (filtered) vs atsr (swansey) black  better worse  green

45. gaat (globaer) vs atsr (swansey) black  better worse  green

46. gaa2 (filtered) vs. gaat (globaer) bb black  better worse  green

47. gaat (globaer) vs. misr  better worse 

48. AODATSR – GlobAER 2004

49. AODSEVIRI – GlobAER 2004

50. AODMERIS – GlobAER 2004

51. AODmerged – GlobAER 2004

52. AODSEVIRI – RUIB 2004

53. AODATSR – Swansea 2004

54. annual maps – AOD 2004 med median cli ‘climatology’ sunsun-photo Globaer products MOMODIS Efcforecast MIMISREasassimilation SEVIRIAATSRMERIS

55. annual maps – Angstrom 2004 med median cli ‘climatology’ sunsun-photo Globaer products MOMODIS Efcforecast MIMISREasassimilation AATSRMERISSEVIRI