Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Aerospace Corporation (Aerospace) Prelaunch Assessment of the Northrop Grumman VIIRS Cloud Mask Thomas Kopp, The Aerospace Corporation Keith Hutchison,

Published byMervyn Charles Modified over 8 years ago

Similar presentations

Presentation on theme: "The Aerospace Corporation (Aerospace) Prelaunch Assessment of the Northrop Grumman VIIRS Cloud Mask Thomas Kopp, The Aerospace Corporation Keith Hutchison,"— Presentation transcript:

1 The Aerospace Corporation (Aerospace) Prelaunch Assessment of the Northrop Grumman VIIRS Cloud Mask Thomas Kopp, The Aerospace Corporation Keith Hutchison, Northrop Grumman Andrew Heidinger, NOAA/STAR Richard Frey, University of Wisconsin IGARSS 25 July 2011

2 2 thomas.kopp@aero.org Meteorological Satellite Systems Outline Definitions of VIIRS Cloud Mask (VCM) contents and validation conditions High level review of the VCM logic Global results with the pre-launch VCM without any tuning Quantitative Improvements Using the Northrop Grumman (NG) tuning tool Methods for evaluating individual granules during Intensive Cal/Val (ICV) of the VCM

3 3 thomas.kopp@aero.org Meteorological Satellite Systems VCM Contents The VCM itself determines one of four cloud cover conditions for each pixel –Confidently Cloudy –Probably Cloudy –Probably Clear –Confidently Clear All downstream EDR products, except for imagery, require the VCM as an input Downstream products will use either the confidently cloudy or confidently clear condition –The probably clear/cloudy cases account for pixels that are not completely cloud covered but due either to the difficulty of the scene or partial clouds such as cumulus, are not sufficiently clear to reliably determine the conditions at the surface

4 4 thomas.kopp@aero.org Meteorological Satellite Systems VCM Performance Metrics Probability of Correct Typing (PCT ) − PCT = (1 - Binary Cloud Mask Error) = {1 – [(VCM = conf. clear) & (Truth = conf. cloudy) OR ((VCM = conf. cloudy) & (Truth = conf. clear)]/[total #pixels in each geographic class – PCPC] Cloud Leakage (CL) − CL = [(VCM = conf. clear) & (Truth = conf. cloudy)]/total #pixels in the geographic class False Alarm Rate (FA) − FA = [(VCM = conf. cloudy) & (Truth = conf. clear)]/total #pixels in the geographic class Fraction of Pixels Classified as Probably Clear/Cloudy (FPCPC) –FPCPC = [(VCM = prob. clear) or (VCM = prob. cloudy)]/total #pixels in the geographic class

5 5 thomas.kopp@aero.org Meteorological Satellite Systems Overview of VCM Approach There are five possible processing paths in the VCM algorithm for the analysis of SDR data collected in daytime conditions Cloud Spectral Tests Used to Determine Cloud Confidence WaterLandDesertCoastSnow 1.M9 (1.38 µm) Reflectance TestXXX (if TPW > 0.25 cm) XX 2. M15-M16 (10.76 – 12.01 µm) Brightness Temperature Difference (BTD) XXXX 3.Tri-Spectral M14, M15, M16 (8.55, 10.76, 12.01 µm) BTD Test X 4.M15-M12 (10.76-3.70 µm) BTD Test X (if no sun glint) X (if TOC NDVI > 0.2) X (if Lat > 60  or < - 60  ) X (if no sun glint and if TOC NDVI > 0.2) X 5.M12-M13 (3.70-4.05 µm) BTD Test X (if –60  < Lat < 60  ) and no sun glint X (if –60  0.2 X (if –60  < Lat < 60  ) 6.M1 (0.412 µm) Reflectance TestX (if –60  < Lat < 60  ) 7.M5 (0.672 µm), M1 (0.412 µm) Reflectance Tests X (M5 if TOC NDVI ≥ 0.2; M1 otherwise) 8.M7 (0.865 µm) Reflectance TestX 9.M7/M5 (0.865 / 0.672 µm) Reflectance Ratio Test XX (if RefM5 ≥ LD_M5_Gemi Thresh) Cloud Spatial Tests Used to Modify the Final Cloud Confidence Classification WaterLandDesertCoastSnow 10.I5 (11.45 µm) Spatial TestX 11.I2 (0.865 µm) Reflectance TestX

6 6 thomas.kopp@aero.org Meteorological Satellite Systems Pre-launch, Pre-tuned Global VCM Results VCM version used 1.5.0.48 from 2009 The initial thresholds were used, the VCM for this testing was not tuned Comparisons made with collocated MOD35 C6 cloud mask and CALIOP matchups for comparison – Cloudy for the VCM in this case included probably cloudy pixels – Clear for the VCM in this case included probably clear pixels Compared only 1-km CALIOP segments with either 0% or 100% cloud cover –Resulted in approximately 15 million collocations per month

7 7 thomas.kopp@aero.org Meteorological Satellite Systems Caveats and Notes to Global VCM Results Results intended to show “where we were” in late 2009 Neither of the two I-band tests could be simulated using the proxy data, a significant source of error that will not be quantified until the post-launch validation of the VCM Thin cirrus has a major impact on the results Analysis limited to near-nadir views (MODIS viewing zenith angle of +/- 20 degrees) Hit rate = (# agree cloud + # agree clear) / total # Hanssen-Kuiper Skill Source (HKSS) = (# agree cloud * # agree clear) – (# disagree cloud * # disagree clear) / (# agree clear + # disagree clear) * (# agree cloud + # disagree cloud) Results follow on the next few slides

8 8 thomas.kopp@aero.org Meteorological Satellite Systems VCM versus CALIOP, Global Results

9 9 thomas.kopp@aero.org Meteorological Satellite Systems VCM versus CALIOP, Polar and Non-Polar Results

10 10 thomas.kopp@aero.org Meteorological Satellite Systems VCM versus CALIOP, Day Results

11 11 thomas.kopp@aero.org Meteorological Satellite Systems VCM versus CALIOP, Night Results

12 12 thomas.kopp@aero.org Meteorological Satellite Systems VCM versus CALIOP, Land and Desert Results

13 13 thomas.kopp@aero.org Meteorological Satellite Systems VCM Designed to Exploit VIIRS 1.38-µm Data MODIS vs VIIRS RSRs MODIS vs VIIRS TOA Radiances VIIRS OOB Response is orders of magnitude less MODIS OOB Response is as large as the in- band response Thin cirrus clouds will be more readily detected with VIIRS data than in MODIS

14 14 thomas.kopp@aero.org Meteorological Satellite Systems VCM Versus Heritage Performance, COT > 1.0 VCM and heritage performance are comparable when thin cirrus clouds are eliminated from the results

15 15 thomas.kopp@aero.org Meteorological Satellite Systems One Year Means of Hit Rates and Skill July 2007 – June 2008, Comparison with CALIOP Scene Type Mean VCM Hit Rate Mean MOD35 C6 Hit Rate Mean VCM HKSS Mean MOD35 C6 HKSS Global79.086.962.072.51 60S-60N84.189.974.078.1 Global day83.489.066.176.3 Global night75.185.159.669.1 60S-60N Water day 87.990.977.779.8 60S-60N Water night 83.290.576.476.0 60S-60N Land day 80.788.263.876.6 60S-60N Land night 80.187.465.074.7

16 16 thomas.kopp@aero.org Meteorological Satellite Systems Pre-Launch Tuning Approach Pre-launch tuning is based on 14 granules which employed Global Synthetic Data (GSD) – Of these 14, 11 contained land backgrounds These granules covered each VCM geographic type and ranged from straightforward to difficult scenes GSD provides unique data to set the mid-point thresholds – Typical methods of tuning, using on-orbit sensor data, rely upon 100% cloudy and 100% cloud free distributions – GSD alone allows cloud distributions to be evaluated at the mid-point (50% cloudy) condition GSD allows setting thresholds and then minimize the distance between the confidently cloudy and confidently clear thresholds

17 17 thomas.kopp@aero.org Meteorological Satellite Systems Advantages Using Global Synthetic Data (GSD) GSD allows testing of the tuning process with proxy data and then apply the procedure to VIIRS-unique data – Tuning process is validated by : – (1) tuning with GSD truth data developed with MODIS Relative Spectral Responses (RSR) – (2) tested in the VCM using MODIS granules – (3) quantitatively evaluated using manually-generated cloud data of the MODIS data – Tuning for VIIRS data is then completed by examining changes in cloud distribution for each test in GSD truth data using the VIIRS RSR

18 18 thomas.kopp@aero.org Meteorological Satellite Systems Initial VCM Results Showed Following Needs Reduce the number of probably clear and probably cloudy (PCPC) classifications by adjusting the overall cloud confidence threshold Identify tests that generated the highest percentage of false alarms for each VCM background condition and tune the mid-point thresholds (i.e. 50% cloud cover condition) accordingly – only possible with GSD Further reduce the number of PCPC classifications, as necessary, by adjusting the distance between the mid-point thresholds of a given individual cloud test and the low and/or high threshold using cloud distributions in the GSD.

19 19 thomas.kopp@aero.org Meteorological Satellite Systems Initial Untuned VCM Performance - Land Granule ID 2001152 _1600 2001196_ 1755 2001213_ 1210 2001213_ 1220 2002001_ 0340 2002032_ 1745 2002032_ 1750 2002118 _0215 2003194_ 0905 2003299_ 1835 2003299_ 1840Summary land 62833611271204503054432136760068944910227706737949375711249880108676 nPoorQual 00000000000 nCldTruth 594640.44605614489261333822034154313934674146538896817132128760 nClrTruth 336966810693054128299985394274018629301259141848603107856079916 nConfCldy 59018246653413811325904864464583384610275396724391024163528341 nConfClr 63512229953182543156780497011708386565645071035247634965 nPrbCldy 2660869210371032543493656323768146853290222541251 nPrbClr 2914342890427000202676700221448473671355378143966163351744119 FalseAlarms 11974462301316772313128682537936871257231378897672630 Leakage 47213462838131353867264138951315111 BinaryError 124464757613447761132599830759435912578715273910822741 FPCPC 0.050.390.620.390.530.260.380.100.470.520.420.38 pFalseAlarms 0.020.070.080.230.200.160.150.210.000.150.040.12 pLeakage 0.00 0.030.00 pBinaryError 0.020.070.080.230.210.160.150.210.030.150.04 PCT 0.980.930.920.770.790.840.850.790.970.850.960.87

20 20 thomas.kopp@aero.org Meteorological Satellite Systems Overview of the Pre-Launch Tuning Process Identify the tests causing the largest number of errors Use GSD with MODIS RSRs to generate cloud cover distributions for the cloud detection tests identified above – Generate distributions for 0%, 50%, and 100% cloud cover – Set key mid-point threshold using the 50% cloud cover, then minimize low- and high thresholds Update VCM using these thresholds Execute the updated algorithm on the set of MODIS granules Evaluate the performance using the manually generated cloud masks Assess the changes in performance

21 21 thomas.kopp@aero.org Meteorological Satellite Systems Example for a Case With Too Many PCPC Pixels MODA.2001.196.1755 Manually-Generated Mask Q thresh = 99% Land - Pre FPCPC0.39 pFalseAlarms0.07 pLeakage0.00 pBinaryError0.07 PCT0.93 Q thresh = 90% Land - Post FPCPC0.14 pFalseAlarms0.05 pLeakage0.01 pBinaryError0.06 PCT0.94

22 22 thomas.kopp@aero.org Meteorological Satellite Systems Specific Cloud Detection Case, GEMI Test (Land) Changed from 1.95 to 1.87Changed from 1.90 to 1.82Changed from 1.85 to 1.78

23 23 thomas.kopp@aero.org Meteorological Satellite Systems Quantitative Impacts – GEMI Results M7/M5 Land62833611271204503054432136760068944910227706737949375711249880108676Summary nPoorQual00000000000 nCldTruth59464044605614489261333822034154313934674146538896817132128760 nClrTruth336966810693054128299985394274018629301259141848603107856079916 nConfCldy58151640838011854205374382294040754335274661033954221106324899 nConfClr18824650863293202188580481718368750509113460888573196017878081 nPrbCldy3268825928390317190239361125212251149011933426 nPrbClr247285962335731110710736177751728986083210808667085270 FalseAlarms6113111291496778520955293709805398721277525984 Leakage211616834154480812483151231039529839374128272674 BinaryError822927963169314859769270416813755428539586903523658 FPCPC0.040.060.090.220.090.150.080.110.010.060.050.07 pFalseAlarms0.01 0.000.010.060.080.070.080.000.060.010.05 pLeakage0.000.010.030.01 0.020.010.000.040.010.020.01 pBinaryError0.010.030.04 0.080.120.09 0.040.080.04 PCT0.990.970.96 0.920.880.91 0.960.920.960.93 Previous untuned results FPCPC0.040.060.460.220.370.220.180.080.120.190.150.17 pFalseAlarms0.01 0.080.110.080.190.000.060.010.06 pLeakage0.000.010.000.010.00 0.040.000.01 pBinaryError0.010.030.020.030.140.150.110.200.040.080.03 PCT0.990.970.980.970.860.850.890.800.960.920.970.91

24 24 thomas.kopp@aero.org Meteorological Satellite Systems Tests Improved by the Pre-Launch Tuning Effort Reflectance test over desert (M1) Reflectance test over land (M5) Reflectance test over water (M7) Ratio test over land (GEMI) Ratio test over water (M7/M5) Mid-Wave minus long wave infrared over snow (M12 – M15) Mid-Wave infrared difference over snow (M12 – M13)

25 25 thomas.kopp@aero.org Meteorological Satellite Systems Performance After Daytime Tuning Completed Performance Measure LandOceanDesertSnow PCPC SysSpec 38.3 15% 19.2 15% 14.7 15% 58.0 15% False Alarms SysSpec 12.4 7.0 6.0 5.010.522.5 Leakage (%) SysSpec 0.5 3.0 0.3 1.02.30.02 PCT (%) SysSpec 87.1 90.0 93.7 94.087.277.4 Performance Measure LandOceanDesertSnow PCPC SysSpec 7.5 15% 22.5 15% 3.5 15% 5.1 15% False Alarms SysSpec 4.6 7.0 1.4 5.02.13.7 Leakage (%) SysSpec 1.4 3.0 0.6 1.03.70.9 PCT (%) SysSpec 93.4 90.0 97.5 94.093.995.1 Untuned VCM: March 2010 Tuned VCM: November 2010

26 26 thomas.kopp@aero.org Meteorological Satellite Systems Tool for Visualization of the VCM The previous analyses reveal quantitative aspects of the VCM, but lack context Historically the capability to visualize the output from each individual cloud detection test has been used operationally at the Air Force Weather Agency Key to a useful visualization are two fundamental factors – It must overlay each test on applicable imagery – It must contain the reflectance/brightness temperatures used within the cloud mask This reveals if any bands have bad or saturated values The visualization should also note if any degraded conditions of note exist in the scene – These include aerosols, sun glint, and shadows The following pair of slides show this capability

27 27 thomas.kopp@aero.org Meteorological Satellite Systems Aerospace Visualization Tool – Example I

28 28 thomas.kopp@aero.org Meteorological Satellite Systems Aerospace Visualization Tool – Example 2

29 29 thomas.kopp@aero.org Meteorological Satellite Systems Conclusion Pre-launch validation of the VCM uses three different approaches to verify the VCM will meet expectations – Large scale quantitative analysis – Small scale quantitative analysis via GSD – Visualization of individual granules with each component cloud detection test Results show promise that the VCM will meet or exceed its requirements Each of these methods will be employed in some form post-launch, though we will no longer need GSD as actual VIIERS data will be available

Download ppt "The Aerospace Corporation (Aerospace) Prelaunch Assessment of the Northrop Grumman VIIRS Cloud Mask Thomas Kopp, The Aerospace Corporation Keith Hutchison,"

Similar presentations

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 A Cloud Object Based Volcanic.

Center for Satellite Applications and Research (STAR) Review 09 – 11 March 2010 Image: MODIS Land Group, NASA GSFC March 2000 A Cloud Object Based Volcanic.
 nm)  nm) PurposeSpatial Resolution (km) 317.51Ozone, SO 2, UV8 3251Ozone8 3403Aerosols, UV, and Volcanic Ash8 3883Aerosols, Clouds, UV and Volcanic.

 nm)  nm) PurposeSpatial Resolution (km) Ozone, SO 2, UV8 3251Ozone8 3403Aerosols, UV, and Volcanic Ash8 3883Aerosols, Clouds, UV and Volcanic.
Improved Automated Cloud Classification and Cloud Property Continuity Studies for the Visible/Infrared Imager/Radiometer Suite (VIIRS) Michael J. Pavolonis.

Improved Automated Cloud Classification and Cloud Property Continuity Studies for the Visible/Infrared Imager/Radiometer Suite (VIIRS) Michael J. Pavolonis.
VIIRS LST Uncertainty Estimation And Quality Assessment of Suomi NPP VIIRS Land Surface Temperature Product 1 CICS, University of Maryland, College Park;

VIIRS LST Uncertainty Estimation And Quality Assessment of Suomi NPP VIIRS Land Surface Temperature Product 1 CICS, University of Maryland, College Park;
Development of a Simulated Synthetic Natural Color ABI Product for GOES-R AQPG Hai Zhang UMBC 1/12/2012 GOES-R AQPG workshop.

Development of a Simulated Synthetic Natural Color ABI Product for GOES-R AQPG Hai Zhang UMBC 1/12/2012 GOES-R AQPG workshop.
Northern hemisphere Fractional Snow mapping with VIIRS - First experiments in ESA DUE GlobSnow-2 Sari Metsämäki 1), Kari Luojus 2), Jouni Pulliainen 2),

Northern hemisphere Fractional Snow mapping with VIIRS - First experiments in ESA DUE GlobSnow-2 Sari Metsämäki 1), Kari Luojus 2), Jouni Pulliainen 2),
VIIRS Cloud Phase Validation. The VIIRS cloud phase algorithm was validated using a 24-hour period on November 10, 2012. Validation was performed using.

VIIRS Cloud Phase Validation. The VIIRS cloud phase algorithm was validated using a 24-hour period on November 10, Validation was performed using.
Cloud Masking and Cloud Products MODIS Operational Algorithm MOD35 Paul Menzel, Steve Ackerman, Richard Frey, Kathy Strabala, Chris Moeller, Liam Gumley,

Cloud Masking and Cloud Products MODIS Operational Algorithm MOD35 Paul Menzel, Steve Ackerman, Richard Frey, Kathy Strabala, Chris Moeller, Liam Gumley,
Geoscience Australia Md Anisul Islam Geoscience Australia Evaluation of IMAPP Cloud Cover Mapping Algorithm for Local application. Australian Government.

Geoscience Australia Md Anisul Islam Geoscience Australia Evaluation of IMAPP Cloud Cover Mapping Algorithm for Local application. Australian Government.
MOD06 Cloud Top Properties Richard Frey Paul Menzel Bryan Baum University of Wisconsin - Madison.

MOD06 Cloud Top Properties Richard Frey Paul Menzel Bryan Baum University of Wisconsin - Madison.
NRL09/21/2004_Davis.1 GOES-R HES-CW Atmospheric Correction Curtiss O. Davis Code 7203 Naval Research Laboratory Washington, DC 20375

NRL09/21/2004_Davis.1 GOES-R HES-CW Atmospheric Correction Curtiss O. Davis Code 7203 Naval Research Laboratory Washington, DC 20375
Multiple Criteria for Evaluating Land Cover Classification Algorithms Summary of a paper by R.S. DeFries and Jonathan Cheung-Wai Chan April, 2000 Remote.

Multiple Criteria for Evaluating Land Cover Classification Algorithms Summary of a paper by R.S. DeFries and Jonathan Cheung-Wai Chan April, 2000 Remote.
16/06/20151 Validating the AVHRR Cloud Top Temperature and Height product using weather radar data COST 722 Expert Meeting 9.-11. 6. 2004 Sauli Joro.

16/06/20151 Validating the AVHRR Cloud Top Temperature and Height product using weather radar data COST 722 Expert Meeting Sauli Joro.
Using satellite-bourne instruments to diagnose the indirect effect A review of the capabilities and previous studies.

Using satellite-bourne instruments to diagnose the indirect effect A review of the capabilities and previous studies.
Calibration Working Group L1b Cal/Val Update 1 Presented by Changyong Cao NOAA/NESDIS/STAR GOES-R Calibration Working Group January 9, 2014.

Calibration Working Group L1b Cal/Val Update 1 Presented by Changyong Cao NOAA/NESDIS/STAR GOES-R Calibration Working Group January 9, 2014.
Shobha Kondragunta 1 and Pubu Ciren 2 GOES-R ABI Aerosol Detection Product (ADP) Validation 1 NOAA/NESDIS/STAR 2 Second AWG Validation Workshop.

Shobha Kondragunta 1 and Pubu Ciren 2 GOES-R ABI Aerosol Detection Product (ADP) Validation 1 NOAA/NESDIS/STAR 2 Second AWG Validation Workshop.
Cooperative Institute for Meteorological Satellite Studies University of Wisconsin - Madison Steve Ackerman Director, Cooperative Institute for Meteorological.

Cooperative Institute for Meteorological Satellite Studies University of Wisconsin - Madison Steve Ackerman Director, Cooperative Institute for Meteorological.
Motivation Many GOES products are not directly used in NWP but may help in diagnosing problems in forecasted fields. One example is the GOES cloud classification.

Motivation Many GOES products are not directly used in NWP but may help in diagnosing problems in forecasted fields. One example is the GOES cloud classification.
Introduction Land surface temperature (LST) measurement is important for understanding climate change, modeling the hydrological and biogeochemical cycles,

Introduction Land surface temperature (LST) measurement is important for understanding climate change, modeling the hydrological and biogeochemical cycles,
CRN Workshop, March 3-5, 2009 1 An Attempt to Evaluate Satellite LST Using SURFRAD Data Yunyue Yu a, Jeffrey L. Privette b, Mitch Goldberg a a NOAA/NESDIS/StAR.

CRN Workshop, March 3-5, An Attempt to Evaluate Satellite LST Using SURFRAD Data Yunyue Yu a, Jeffrey L. Privette b, Mitch Goldberg a a NOAA/NESDIS/StAR.

Similar presentations

Ads by Google